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CHAPTER ONE 
OVERVIEW OF THE EVALUATION 



This initial chapter provides an overview of both the CAUSE program 
and this evaluation effort. The chapter begins with a description of the 
operation and history of the CAUSE program, A concluding section outlines 
the development and general design of the evaluation. 

Description of the CAUSE Program 

According to the 1979 program announcement the Comprehensive 
Assistance to Undergraduate Science Education (CAUSE) program encourages 
the improvement in quality and effectiveness of undergraduate science 
education in institutions of higher education. The primary objectives of 
the CAUSE program are to: (1) strengthen the resources for undergraduate 
science education components of 2-year and 4-year colleges and universities 
(2) improve the quality of science instruction at the undergraduate level; 
and (3) enhance the capability of institutions for self-assessment, manage- 
ment, and evaluation of their science programs. 

Any nonprofit, degree-granting institution or consortium of institu- 
tions in the United States is eligible to submit a proposal. Program 
objectives are to be achieved through projects identifying and meeting 
local science education needs. CAUSE is intended to provide for compre- 
hensive revitalization of an instructional program either within or across 
departments. Funds cannot be requested for major construction or addi- 
tional faculty except as required for improving an instructional program. 
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Support is intended to result in improvements which will continue beyond 
the actual funding period. Grants may be requested for up to $250,000 
over a 3-year period. The proposing institution must provide at least 

one-third of the total project costs. 

Any physical or social science discipline is eligible for CAUSE funds 
as are interdisciplinary efforts in the sciences. Each institution is 
allowed to submit only one proposal. At the institutional level this 
necessitates the development of some procedure, formal or informal, for 
determining which department or group of departments will submit a pro- 
posal to the CAUSE program. 

Proposals are grouped for review into categories of 2-year colleges, 
baccalaureate-granting institutions, Ph.D. -granting institutions and 
consortia. A consortium may be formed for the sole purpose of planning a 
project which will benefit several institutions. A system of peer review 
has been established for assessing the proposals. The majority of peer 
reviewers assigned to read proposals from baccalaureate-granting institu- 
tions are themselves members of such institutions. The same is true for 
the other institutional types. Similarly, biologists tend to be assigned 
to read biology proposals, physicists to read physics proposals, and so on. 

The CAUSE program was founded by an act of Congress in 1975 and 
awarded its first 59 grants on June 18, 1976. It has just completed its 
fourth year. 

A Profile of One Year of the CAUSE Program 

As an example of projects funded in one year by CAUSE a profile of 
1979 projects is presented. In the current year NSF received a total of 
307 proposals and funded 72 of these. Table 1 portrays proposals received 
compared to proposals funded according to institutional type. 

1Q 
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Table 1 

Summary of Requests and Actions 
CAUSE FY 1979 



Institutional Proposals Amounts Projects Success Award Funds 

Type Received Requested Recommended Ratio (%) . Requested Recommended 



Two-Year 


68 


$11,165,340 


16 


23.5 


$2,612,626 


$2,523,467 


Four-Year, 
non Ph.D. 


128 


22,206,700 


30 


23.4 


4,986,696 


5,104,775 


Ph.D. 


100 


19,737,409 


23 


23.0 


5,218,010 


5,149,342 


Consortia 


11 


2,271,128 


_3 


27.3 


549,847 


521,859 


TOTAL 


307 


$55,380,577 


72 


23.5 


$13,367,179 $13,299,443 



Broken out by distribution of proposals submitted and funded by 
region of the United States, FY '79 funding looks like this: 

Table 2 

Distribution of Proposals Recommended/Received by Census Region 



CAUSE FY 1979 





Number of Proposals 


Success 


Census Region 


Received 


Awarded 


Ratio (%) 


I New England 


21 


4 


19.0 


II Middle Atlantic 


61 


18 


29.5 


III East North Central 


44 


9 


20.5 


IV West North Central 


28 


8 


28.6 


V South Atlantic 


56 


12 


21.4 


VI East South Central 


23 


5 


21.4 


VII West South Central 


28 


6 


21.4 


VIII Mountain 


20 


3 


15.0 


IX Pacific 


26 


_7 


26.9 


TOTALS 


307 


72 


23.5 



The 1979 proposals portrayed by institution type according to groups 
of students served by the proposed project are shown in Table 3. 

Table 3 

Number of Students/Year Involved in Awards 
By Type of Institution 
CAUSE 1979 

Students/Year Minority Women Handicapped 

Type No. % ' No. % No. % No. % 

2-Year 9,520 16.5 2,020 21.2 4,150 43.6 280 3.0 
4-Year 

non-Ph.D. 22,110 38.3 5,090 23.0 11,470 51.9 210 1.0 

Ph.D. 21,580 37.3 2,030 9.4 9,580 44.3 280 1.2 

Consortia 4,580 7.9 530 11 .5 2,150 46.9 _50 K£ 

TOTAL 57,790 100.0 9,670 16.7 27,350 47.3 820 1.4 



Across Project Years 

Between 1976 and 1979, the four years that the CAUSE program has been 
in existence, a total of 273 projects have been funded. Numbers of projects 
increased somewhat from 1976 to 1978 (1976 = 59, 1977 = 69, 1978 = 73) and 
levelled off in 1979 (72). Similarly, the amount of money awarded by NSF 
increased gradually from 1976 to 1978 (1976 = $9.9 million, 1977 = $10.7 
million, 1978 = $13 million). Table 4 portrays the number and amount of 
awards by institution type. 
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Table 4 

Proposals Received, Awarded and Amount 
CAUSE 1976-1979 

1976 1977 1978 1979 Total 



Type 


Red. 


Awarded 




Red. 


Awarded 


z: 


•o 

or 


Awarded 
$M 


•o 
o 
or 


Awarded 
$M 


Red. 


Awarded 


2: 


Two-Year 


186 


17 


2.3 


105 


19 


2.6 


82 


20 3.1 


68 


16 2.5 


441 


72 


10.5 


Four- Year 


372 


23 


3.3 


211 


28 


4.6 


137 


34 6.0 


128 


30 5.1 


848 


115 


19.0 


Ph.D. 


157 


17 


3.9 


138 


16 


3.0 


111 


17 3.5 


100 


23 5.2 


506 


73 


15.6 


Consortia 


45 


2 


0.4 


29 


6 


0.5 


16 


2 0.4 


11 


3 0.5 


io; 


13 


1.8 


TOTAL 


760 


59 


9.9 


483 


69 


10. 7 


346 


73 13.0 


307 


72 13.3 1896 


273 


46.9 



What Table 4 illustrates is that the number of proposals submitted 
has dropped steadily each year, while the number awarded has risen each 
year until 1979, when the number of awards leveled off. On the next page, 
Table 5 portrays the success rate of institutions which resubmit proposals 
to CAUSE after being turned down initially. This table suggests that the 
likelihood that an institution's project will be funded improves on re- 
submission. The success rate for first-time submissions ranges from 8.1% 
to 16.9%; second submissions are successful 16.8% to 23,6% of the time; 
and third and fourth tries have success rates of 26% and 25% respectively. 
Considered together, the evidence of Tables 4 imd 5 suggests that the 
CAUSE program is providing good funding opportunities to higher education 
institutions. 



Table 5 

Success Rates by Number of Resubmissions Per Institution 

CAUSE 1976-1979 



Mew 2nd Try 3rd Try 4th Try Total 




1976 715 57 8.1 715 57 8.1 

1977 133 9 6.8 321 54 16.8 454 63 13.9 

1978 98 13 13.3 89 21 23.6 143 37 25.9 330 71 21.5 

1979 83 14 16.9 94 22 23.4 56 T5 26.8 64 16. 25.0 290 67 23.1 
TOTAL 1029 93 9.0 504 97 19.2 199 52 26.1 64 16 25.0 1789 255" 14.4 



a Data does not include proposals from consortia, or proposal s/awards 
from/to previous awardees. A resubmission is any proposal from an 
institution with a prior year submission, without regard to content 
match. 



Table 6 indicates that the number of institutions submitting proposals 
for the first time has dropped over the four years that CAUSE has been in 
existence. Given the newness of the program, however, little significance 
can be attached to this drop, particularly since the number of first-time 
submissions has remained a constant proportion of the total number of sub- 
missions over the last three years. 



Table 6 



New Institutions 1976 1977 1978 1979 TOTAL 

(not including 

Consortia) 715 133 98 83 1029 



Across project initiation years, the relative percentages of funds 
given to different types cf institutions has been stable. Table 7 portrays 
the number and percentage of awards given to institutions categorized 
according to the Carnegie classification system. The left side of the 
table lists the number and percent of institutions and their enrollments by 
Carnegie classification as they existed nationally for 1979. The right 
side of the table presents the total number and percent of CAUSE awards to 
institutions by classification for the period 1976-1979. 



Table 7 

CAUSE Awards (1976-1979) by Type of Institution 
Using Carnegie Classification 





All 


Institutions for 1979 




CAUSE Awards 


1 976-1 979 a 




Institutions 
Number % 


Enrol lment 
Total % 


Number 


% 


Doctoral Granting 
Institutions 


184 


7 


3,056,132 


28 


53 


20 


Comprehensive 
Universities 
and Colleges 


594 


23 


3,169,495 


29 


74 


29 


Liberal Arts 
Colleges 


583 


23 


531 ,174 


5 


56 


22 


Two-Year Colleges 
and Institutes 


1,147 


45 


3,978,034 


37 


70 


27 


Schools of 
Engineering and 
Technology 


46 


2 


69,508 


1 


5 


2 


TOTAL 


2,554 


100 


10,804,343 


100 


258 


100 



13 Consortia and 2 unclassified awards not included. 
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If we use the total percentage of students enrolled as the criterion 
for the proportion of awards to be given to each category of institution, 
it seems to be the case that liberal arts colleges are a bit over- 
represented, having received 22% of the awards with only 5% of the students, 
while two-year colleges, with 37% of the total student population, have 
received only 27% of the CAUSE awards. 

Minority institutions, as portrayed in Table 8, have consistently 
been awarded CAUSE grants at about the same rate of success as non-minority 
institutions. 



Table 8 
Minority Institutions 
CAUSE 1977-1979 



Year 


No. of Proposals 
Received Awarded 


Success 
Minority Inst. 


Ratio 
All 


Proposals 


1977 


27 


4 


14.8% 




14.1 % 


1978 


26 


6 


23.1 




22.1 


1979 


29 


7 


24.1 




23.5 



All physical and social science disciplines are eligible for CAUSE 
monies. As shown in Table 9, the most frequent type of proposal submitted 
has been multidisciplinary, with proposals from biology a distant second. 
The least frequently submitted proposals have been from earth science 
followed closely by proposals from physics. 
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Table 9 

Proposals Received/Awarded by Science Discipline 
CAUSE 1976-1979 



FY 1979 FY 1976-1979 

No. of Proposals No. of Proposals 



Discipline 


Received 


Awarded 


Success 
Ratio 


Received 


Awarded 


Success 
Ratio 


Multiple Disci pi ine 


178 


43 


24.2% 


1165 


162 


13.9% 


Biology 


29 


7 


24.1 


231 


39 


16.9 


Chemistry 


16 


5 


31.3 


99 


16 


16.2 


Engineering 


23 


5 


21.7 


101 


15 


14.9 


Earth Science 


5 


2 


40.0 


30 


4 


13.3 


Mathematics 


28 


6 


21.4 


104 


15 


14.4 


Physics 


8 


0 


0.0 


53 


7 


13.2 


Social Science 


20 


_4 


20.0 


113 


15. 


13.3 


TOTAL 


307 


72 


23.5 


1896 


273 


14.4 



History and Background of CAUSE 

Origins 

CAUSE is a fairly young program; it is the youngest of the three 
NSF programs now aimed at improving college science programs (the other 
two are ISEP, Instructional Science Equipment Program and LOCI, Local 
Course Improvement Program). However, CAUSE has its origins in the College 
Science Improvement Program (COSIP) which began in 1967. This was a pro- 
gram whose goal was to broaden the availability of superior science pro- 
grams in all regions of the United States. Specifically, COSIP was re- 
sponsive to the needs of 2-year, 4-year and minority institutions to 
improve their science programs at a time when most grant programs were 
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aimed at the great research universities- In 1973, however, cuts in 
NSF's budget forced a discontinuation of the COSIP program. 



Congressional Mandate for CAUSE 

When the National Science Foundation presented its Fiscal Year 
1976 budget to Congress in the spring of 1975, no mention was made of 
any broad undergraduate science program. CAUSE resulted from hearings 
on the NSF budget by the Subcommittee on Science, Research and Technology, 
U. S. House of Representatives. The specific instigation was public testi- 
mony in which the need for support for the improvement of undergraduate 
science education was stressed. The 1976 NSF Authorization Act, Public 
Law 94-86, contains this section pertaining to CAUSE: 

The National Science Foundation is authorized and directed 
to conduct a Comprehensive Assistance to Undergraduate 
Science Education program, referred to hereinafter as 
CAUSE. CAUSE shall have the purpose of strengthening the 
science education capabilities of predominantly under- 
graduate educational institutions and departments or 
groups of departments thereof through awards to four-year 
colleges, two-year colleges, to the undergraduate component 
of advance degree institutions, and to groups of such 
institutions. . . 



Purposes and Objectives of the Program 

The Committee Conference of the two Houses reviewed the purpose of 

CAUSE and its potential role within the overall NSF science education 

program. The Conference Report includes this statement describing 

congressional intent as it related to CAUSE: 

The CAUSE program, initiated by the committee for the coming 
fiscal year, will provide a means of strengthening under- 
graduate science education in the Nation's colleges. This 
program will, provide specialized science teaching equipment, 
science teaching materials, and will offer opportunities 
for the implementation of new methods of teaching science 
developed elsewhere, and will permit departments and institu- 
tions to develop their own approaches to science teaching. 
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CAUSE will also provide funds for the further training of 
science faculty and for the addition of new science faculty. 
Grants will be made on a competitive basis . . . Interdis- 
ciplinary approaches to teaching of science will be encouraged. 
Particular emphasis will be placed on encouraging the teaching 
of science in minority institutions, and in two-year colleges. 

In the final NSF Appropriations Act of 1976, $2.3 million out of a total 
budget of $10 million was set aside for 2-year institutions. 

The Conference Report does not explicitly define the term "compre- 
hensive". NSF has offered this definition in contrast to its other 
programs: 

. . . CAUSE, however, offers a comprehensive approach to the 
improvement of an institution's instructional programs by 
support of broader, more integrated projects. Therefore, a 
CAUSE proposal must be built upon the application of a set of 
coordinated activities . . . These activities are expected to 
improve science education in a particular department or a 
larger unit within the institution. (CAUSE Program Announcement, 
1978) 

In summary, the Congressional intent in establishing the CAUSE program 
was to: (1) strengthen undergraduate science education; (2) provide 
integrated and broadly-based science improvement programs at the institu- 
tional level; (3) provide for faculty development; (4) encourage inter- 
disciplinary approaches to the improvement of science programs; (5) place 
particular emphasis on improvement of science education at minority and 
2-year institutions. 



Description of the Evaluation of CAUSE 



History and Background of the Evaluation 

In the summer of 1977, the Office of Program Integration at the 
National Science Foundation invited interested organizations to submit 
proposals for the evaluation of CAUSE and several other programs. The 



19 



12 



proposal competition was organized in two phases. The first phase 
was the awarding of three planning grants of $10,000 each for six 
months to develop evaluation designs. The second phase was the awarding 
of funds to support the execution of the evaluation by the group which 
produced the most promising design in the first phase. By issuing a 
general program solicitation rather than resorting to the RFP mechanism, 
NSF hoped to encourage more imaginative evaluation designs. 

In its Program Evaluation Solicitation , NSF defined the purpose 
for the evaluations as helping to ". . . generate information on the 
need, scope, impact, and conduct of the programs in addition to indicating 
possible future implications, trends, policies, and theoretical issues." 
The primary audience for the evaluations was to be the National Science 
Foundation and secondary audiences were to-be Congress, the executive 
branch and academic and professional audiences. 

The Solicitation listed 10 questions which were defined as "possible 



concerns 


that might be raised." These questions are: 


1. 


What is the nature and scope of the need for the program? 


2. 


What rationale can be constructed for program activities? 


3. 


What are the characteristics that lead to success or failure 




for the program? 


- 4. 


Is the program achieving its objectives? 


5. 


How well does the program meet the needs identified? 


6. 


What promising practices or products have been developed? 


7. , 


What are the indirect outcomes of the program? 


8. 


What lasting effects has the program achieved? 


9. 


What indications of causality are there? 


10. 


What policy implications are indicated? 
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These questions were general to all the NSF programs to be evaluated. 
Specifically in reference to CAUSE, the Solicitation states: "As the 
CAUSE program is quite new, evaluation must necessarily focus on ante- 
cedents, program rationale, project activities and intermediate outputs." 
Specific questions listed in the Solicitation were: 

1. Nature and scope of need for CAUSE: 

a. What evidence exists that 2- and 4-year institutions 
need Federal assistance to keep pace with advances 

in science knowledge or improvements in science teaching? 

b. What are the specific areas of undergraduate. science 
education most in need of Federal assistance? .Why? 

2. Does the CAUSE program adequately respond to these needs? 

a. Do eligible institutions consider the CAUSE program a 
viable and efficient mechanism for improvement? 

b. What impact (direct or indirect) might the present 
CAUSE program have on the target populations? On 
funded institutions? 

3. Project outcomes 

a. What are the factors within projects that lead to success 
or failure? 

b. What preliminary innovative or promising practices can 
be found in CAUSE projects? 

c. What baseline information should be collected for later 
evaluation comparisons? 

d. Given the early nature of these projects, is there 
evidence to suggest the projects are meeting the program 
objectives? 

4. Are there policy changes and program modifications the CAUSE 
program should consider? 

NSF's thrust in these questions and issues was to elicit information 
about the needs, implementation strategies, outcomes and potential program 
modifications as these were relevant at the institutional level. They 
were not concerned that the evaluation focus on NSF's internal decision- 
making, policies, procedures or selection of awards since these were the 
purview of external oversight activities. Dr. Richard Atkinson, Director 
of the National Science Foundation, wrote a memorandum in 1977 which 
differentiated between the external oversight function then being 
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established and program evaluation. He said, in part: 

Evaluation deals with the results of NSF research, science 
education, and other support activities. It is expected 
that program oversight and program evaluation will provide 
information that will be helpful in setting priorities and 
in assuring accountability and effective program management 
at all levels of organization. 



The PEA Evaluation 

In responding to the needs and priorities identified by NSF, 
Development and Evaluation Associates, Inc, designed an evaluation which 
would: (1) be responsive to the wide diversity in CAUSE projects at the 
institutional level; (2) be formative in nature, facilitating the refine- 
ment and improvement of the CAUSE program; (3) be comprehensive in its 
scope and at the same time provide detailed examination of a few CAUSE 
projects. 

Out of the issues identified by NSF, we formulated six which we 
believed would potentially yield the most useful information: 

1, To what extent are high priority institutional needs being 
met by local CAUSE projects? 

2, How are the CAUSE projects being implemented? What are their 
strengths and weaknesses? 

3, To what extent is the improvement of the quality of instruction 
occurring as a result of CAUSE? 

4, What is the nature and quality of the evidence and the 
evidence collection and analysis procedures being used to 
determine the strengths and weaknesses of individual 
CAUSE projects? 

5, What are the relative costs of the design, implementation, 
and operation of activities within CAUSE projects, and how 
do they relate to post-CAUSE institutional support? 

6, What policy changes and recommendations should be implemented 
in the CAUSE program? 
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In seeking ways to be both comprehensive and provide detailed 
examinations of some CAUSE projects, we designed a three-pronged 
evaluation (see Figure 1). 

The broad focus evaluation . This component of the evaluation 
provided information on all CAUSE projects funded between 1976 and 1978. 
It included an examination of proposals submitted to NSF and subsequently 
funded and a survey of all project directors. 

The medium focus evaluation . This component consisted of one-time 
only site visits to 17 CAUSE projects in order to provide a look at 
the projects from the participants' point of view and provide some 
insight on how CAUSE projects were being implemented. 

The narrow focus evaluation . This component of the evaluation 
provided an in-depth view of eight CAUSE projects with multiple site 
visits to each. A detailed understanding of _ these projects was made 
possible by this approach. The cost issue was explored in connection 
with these case studies. 

This evaluation was thus designed around three independent studies 
using methods appropriate to the three levels of focus. Data was 
collected and analyzed separately. The data and results of the medium 
and narrow focus evaluation efforts are reported in Volume II of this 
report. The findings and analysis of the broad focus activities are 
presented in Volume III. Chapters Three and Four of this volume present 
conclusions and recommendations derived from all three foci of the 
evaluation. The design and conduct of this evaluation is explained in 
detail in the next chapter. 
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CHAPTER TWO 
METHODOLOGY OF THE EVALUATION 

This chapter describes the methodology of the evaluation. The 
elements of the design of the evaluation include the issues, the evalua- 
tion framework, data collection techniques and activities, and data 
analysis. The part of the chapter which focuses on the data collection 
techniques and activities describes site selection, site visits, case 
studies, survey of project directors and analysis of proposals. 



Issues 



Source of the Issues 

Six issues serve as criteria to focus this evaluation. The primary 
purpose of the evaluation is to provide NSF with a description of the 
CAUSE program in terms of: 

1. To what extent are high priority institutional 
needs being met by local CAUSE projects? 

2. How are the CAUSE projects being implemented? 
What are their strengths and weaknesses? 

3. To what extent is improvement to the quality 
of instruction occurring as a result of CAUSE? 

4. What is tRe nature and quality of the evidence and . 
the evidence collection and analysis procedures 
being used to determine the strengths and 
weaknesses of individual CAUSE projects? 

5. What are the relative costs of the design, 
implementation and operation of activities 
within CAUSE projects, and how do they relate 
to post-CAUSE institutional support? 



6. What policy changes and program modifications 
should be implemented in the CAUSE program? 
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The issues were derived from the original NSF solicitation to ' 
conduct this evaluation, an examination of documents about the program, 
and site visits to a small number of projects. Informal site visits 
were made to eight CAUSE projects during the planning phase of this 
evaluation by an educational evaluator or a science educator. Each 
interviewed project directors and staff, students, administrators, and 
non-CAUSE science faculty. Interviewees were asked to describe their 
knowledge of and experiences with the project at their institution. 
Project staff were asked to describe project successes and failures, to 
suggest ways to improve CAUSE, and to recommend the types of information 
most important to gather in an evaluation of CAUSE. The site visitors 
prepared reports of their visits which summarized the interviews and 
analyzed the site visitor's perceptions of the project. 

We, DEA staff, then distributed the reports and other information 
on CAUSE to an evaluation advisory panel assisting us in designing the 
evaluation. Panel members included educational evaluators, instructional 
technologists, and science educators. Each read the documents and we met 
to discuss the CAUSE program. Our goal was to select a focus and framework 
for the evaluation. Together -we generated a list of more than a hundred 
issues we thought related to the CAUSE program and individual CAUSE projects. 
We made no attempt to judge the significance of each potential issue during 
our discussions. Each issue was just listed as it was generated in order 
to explore as widely as possible the characteristics of the CAUSE program. 

From our discussions we created a list of issues which we circulated 
to the advisory panel. Each panel member was asked to rate each issue 
according to the following guidelines: 
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1. The likelihood that an investigation of the issue 
would lead to improvement of the CAUSE program. 

2. The importance of the issue, as likely to be per- 
ceived by the major audiences of this evaluation. 

3. The specificity of the issue (i.e., would its 
investigation yield information focused enough 
to inform decisions?). 

4. The likelihood that an investigation of the issue 
would contribute to an accurate and comprehensive 
description of the CAUSE program. 

5. The availability of resources to address the issues. 
Based on the ratings, we sorted and analyzed the issues. Ones which 

received low ratings from a majority of the panel were deleted. Ones 
which received high or mixed ratings were analyzed according to the focus 
of the issue. Similar or related issues were grouped and assigned category 
headings. Then we discussed the issues again and reorganized them slightly. 

The resulting categories became the six issues which have provided the 
structure for this evaluation. The issues generated by the advisory panel 
became sub-issues to explain, clarify, and extend the meaning of the six 
issues. Our intention was that the sub-issues be changeable throughout 
the evaluation. In this way we planned for findings to be organized at 
the same time as they informed the evaluation. We wanted to allow the 
evaluation enough scope in order to explore new sub-issues which might 
emerge during the course of the study. The sub-issues with which we 
started are listed below according to which issue they were originally 
assigned during the planning phase. 

A List of the Six Issues and Sub-Issues 

Issue #1 : To what extent are high priority institutional needs being 
met by local CAUSE projects? 



Sub-Issues 

1.1 What are the needs for undergraduate science education 
today? 

1.2 Can the institutional needs claimed in the proposal be 
verified? 

1.3 To what extent are CAUSE project goals integrated with 
institutional goals? 

1.4 What evidence is there that CAUSE projects would not 
have come about without NSF support? 



Issue #2 : How are the CAUSE projects being implemented? What are 
their strengths and weaknesses? 

Sub-Issues 

2.1 What are the proposed objectives of the project? 

2.2 How do they compare with the present objectives? 

2.3 Are the objectives being met? 

2.4 How do the projects "operate?" What activities are 
involved in each? 

2.5 Within the larger institutions, what types of people 
are awarded the CAUSE grants? The best? The worst? 

2.6 What is the evidence that individual CAUSE projects 

have institutional support, both vertically and horizontally 

2.7 How are projects most successfully integrated into 
their respective institutions? To what extent is 
integration evolutionary? 

2.8 What is the nature of CAUSE project planning? How 
carefully is it being done? 

2.9 What is the role of management, administration and 
communication in CAUSE projects? 

2.10 What are the critical attributes of successful and 
unsuccessful CAUSE projects? 

2.11 Who might be labeled as "change agents" in the project? 
What did they do? What are critical attributes of 
change agents in the CAUSE context? 
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2.12 What are the most common constraints on change in 
the CAUSE context? How are they overcome? 

2.13 What evidenceHs there that CAUSE serves as a 
catalyst for change? 

2.14 How does the CAUSE project respond to the instructional 
needs of various student populations? (Science majors, 
non-science majors, ages, ability levels, etc.) 

2.15 To what extent is access to traditional and non-traditional 
student populations being changed as a result of CAUSE? 

I ssue #3 : To what extent is improvement of the quality of instruction 
occurring as a result of CAUSE? 

Sub-Issues 

3.1 What general impact is the CAUSE program having upon 
, undergraduate science programs? 

3.2 What evidence is there that the quality of instruction' 
has improved as a result of CAUSE? 

3.3 What are the affective outcomes of CAUSE (student 
attributes, faculty morale, provincialism, etc.) 

3.4 What unexpected outcomes have occurred especially 
across projects? 

3.5 Are processes, materials, products of instructional 
development efforts of high quality? 

3.6 Are the faculty/instructional developers given adequate 
materials and consulting resources to do their job? 

3.7 To what extent are the materials/products developed 
actually utilized? 

3.8 What events/incidents are seen as critical to project 
successes and failure? Which, if any, of these are 
common across projects? 

3.9 What evidence is there that the effects of the CAUSE 
grant will continue after the funding runs out? What 
are the impacts over time? 

3.10 What preliminary innovative or promising practices 
can be found? 
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Issue #4 : What is the nature and quality of the evidence and evidence 
collection procedures being used to determine the strengths 
and weaknesses of individual CAUSE projects? 

Sub-Issues 

4.1 What role does evaluation play in individual CAUSE 
projects and within the CAUSE program as a whole? 

4.2 How do CAUSE project directors perceive the role of 
evaluator? 

4.3 To what extent are the capabilities of institutions 
for self-assessment and evaluation being improved 
through the CAUSE program? 

4.4 What are the side-effects, the costs and benefits of 
the process of project evaluations? 

4.5 How can project evaluation be improved? 

4.6 What baseline data should be collected for later comparisons: 



Issue #5 : What are the relative costs of the design, implementation anc 
operation of activities within CAUSE projects, and how do the 
relate to post-CAUSE institutional support? 

Sub-Issues 

5.1 What are the costs of design, implementation and operation 
of CAUSE projects? 

5.2 How do design, implementation and operational costs relate 
to post-project institutional support? 

5.3 How are capital expenditures used? What are the returns? 

5.4 What is the effect of an institution's fixed funds 
being appropriated as matching funds for a CAUSE project? 

5.5 What are the implications of CAUSE 's requirement that 
one- third of the total project costs be borne by the 
requesting institution? 

5.6 Are there any consistent differences in the objects of 
expenditures of the two funding sources? 
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Issue What policy changes and program modifications should be 
implemented in the CAUSE program? 

Sub-Issues 

6.1 How can program administration be improved? 

6.2 How can communication with and support of project monitors 
be improved? 

6.3 How can CAUSE project evaluation procedures be improved? 

6.4 How can the documentation and reporting of CAUSE projects 
be improved? 

6.5 What are the strengths and weaknesses of the sampling of 
individual projects? 

6.6 What are the policy implications of the feedback concerning 
the previous five issues? 

Description of the Issues 

Institutional needs , One of the major objectives of the CAUSE program 

is to promote projects which meet high priority local needs of institutions. 

The first part of a CAUSE proposal is a local review statement from an 

institution's chief officer' or chief academic officer. The program 

announcement for CAUSE projects states: 

A requirement for CAUSE support is that the proposed 
activity not only contribute to the improvement of 
instruction in science, but that it do so through 
projects that are consistent with the local environ- 
ment, i.e., the individual institution's overall 
science education objectives (CAUSE Program Announcement, 1979) 

It goes on to specify that an institution should commit itself to a project 

which will fit in the institution; and which can be supported during and 

after the project. Institutions are cautioned to consider priority of 

science education needs. 
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Following the Local Review Statement proposals usually address the 
characteristics of the institution, science education needs, and the choice 
of project objectives to meet the needs. Proposers are also expected to 
describe specifically how the project will be supported after NSF funding 
is over. 

The intent of the Local Review Statement and the discussion of 
institutional needs and project objectives is to identify projects most 
likely to improve science education at the local level. The assumption 
that projects which meet high priority needs- are more likely to succeed 
is not discussed outright. 

During the planning phase of this evaluation the validity and priority of 
local science education needs, as addressed by CAUSE, were examined. We 
assumed at the beginning that institutional needs statements were not 
always sincere and that institutional support might be less than complete. 
Planning phase site visits indicated that CAUSE projects might be primarily 
designed around individual faculty or departmental concerns with little 
regard for institutional concerns. It was because of this preliminary 
evidence and the relative importance given institutional need and commitment 
in proposal review criteria that we selected Issue One. *^ 

During this evaluation we have sought to clarify needs described in 
proposals and judge the degree to which project designs match their stated 
needs. We have tried to determine the extent to which actual institutional 
support for projects exists; where support was lacking we sought to uncover 
the reasons and the consequences of the lack. We also studied whether or 
not a project influenced institutional needs or caused them to be examined 
further. 
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Project implementation . The carrying out of a project is supposed 

to be described in project proposals. Review criteria of the proposal 

include completeness and clarity of the project plan; the relationship 

between objectives and activities; and the sequence of and time allowance 

for project activities. Management of projects is also considered important. 

Proposals should describe management procedures, staffing, and monitoring 

of the project. ' The program announcement recommends that: 

Proposals should be as specific as possible in the 
discussion of how the various proposed activities 
will be carried out so that the feasibility, prac- 
ticality, and potential gain of the project can be 
assessed by reviewers. 

We selected project implementation as an issue to be studied in this 
evaluation because proposals are often scanty in their descriptions of 
proposed-project activities. A major emphasis of this evaluation has 
been to describe local institutional responses to science education needs. 
We sought to compare proposed objectives and activities with those 
actually carried out and to examine the reasons for discrepancies when 
they occurred. We assumed that while successful project implementation 
strategies would differ from site to site, some strategies would be shared. 
We focused specifically on planning, communications, and management as 
potential sources of useful strategies. We thought that future projects 
might benefit from information about such strategies. 

Quality of instruction . Two primary objectives of the CAUSE program 
are to strengthen the nation's undergraduate science education resources and 
to improve the quality of undergraduate science instruction. We recognize 
that these are long-term goals but chose to study as carefully as possible , 
the impact of projects on the quality of instruction. As the program 
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announcement points out: 

Since the ultimate target is the undergraduate student, 
proposals should indicate how the project will have an 
impact on the student's understanding of science. 

The data from the planning phase of the evaluation indicated a wide 
range in the quality of instructional materials and strategies being 
developed on projects. We thought quality might be affected by resources, 
such as instructional development expertise, or special materials available 
to CAUSE faculty for instructional improvement. We knew we would have 
difficulty exploring this issue since assessment of quality of instruction 
of a project is dependent on the evaluation practices of that project. 
We did not intend to make independent assessments or examinations of 
projects or even to conduct secondary analyses of already available data. 
We have examined improvement of science education resources and of quality 
of instruction by studying the processes used, the improvement over what 
was replaced, the probable longevity of the improvement, and the unantici- 
pated outcomes. 

Evaluation , A major objective of the CAUSE program is M to enhance 
the capability of institutions for self-assessment, management, and 
evaluation of their science programs (CAUSE Program Announcement, 1979)," 
The guidelines suggest that up to 10% of the total project budget may be 
expended on evaluation activities. In 1976, 1977, and 1978 the program 
announcement described the roles evaluation can play in a project and 
offered an extensive bibliography on current evaluation theory and methods. 
In 1979 the bibliography was deleted, but prospective project staff are 
encouraged to include two forms of evaluation in their projects: 
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The first should be designed to provide the project's 
staff with whatever information is needed to monitor 
and adjust the project's progress toward its stated 
goals and objectives. The primary beneficiaries of 
this evaluation are the project's staff ... . The 
second form of evaluation should be designed to 
provide the project and other interested parties 
with an overview of project outcomes. (CAUSE 
Program Announcement, 1979.) . 

Evaluation has thus been highly recommended as an important project acti- 
vity for both its format'ive and summative roles. 

Because institutional capacity for evaluation is a goal of CAUSE and 
because evaluation is considered an important project activity we chose to 
study evaluation as our fourth issue. During the planning phase we 
gathered conflicting data on project evaluation. CAUSE documents placed 
a heavy emphasis on it while the projects we visited varied greatly in 
conduct of evaluation activities. We speculated that the conflicting 
evidence might have come from a lack of familiarity with socicil science 
techniques or from a mistrust of the validity or utility of such procedures. 
We also considered that we were not recognizing the fact that most science 
educators routinely collect evic^nce about the success of their teaching 
activities. However, it seemed to us that some project directors were 
more concerned about projeqt activities other than evaluation. During 
our evaluation we have examined evaluation strategies used on projects, 
staff perceptions and knowledge about evaluation, impact of evaluation 
data on projects, and the general nature of problems with evaluation. 

Project costs . Projects are funded with resources from both NSF 
and their own institutions in a two- thirds/one- third match. The program 
announcement notes that: 
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Since CAUSE is intended to provide assistance to 
institutions in improving their own programs of 
science education, or in enhancing their own 
capabilities to conduct such improvement activities, 
recipients of NSF support under this program will 
be required to contribute to overall project costs. 
(CAUSE Program Announcement, 1979.) 

Institutions have great leeway in selecting which budget items they will 
support. Types of items supported by an institution are not factors judged 
in reviewing proposals. Institutional contributions may incUide indirect 
costs at the standard rate for the institution. The institutional con- 
tribution gives evidence to the institutional support promised in the 
local review statement. 

We selected an examination of project costs and resources as Issue 
Five because we believed that costs are closely tied to understanding 
institutional support and continuation of the project after CAUSE. Inves- 
tigations during the planning stage at one site indicated that institutional 
support could be far greater than that promised and allocated in the budget. 
It seemed possible in some cases that NSF funds generated, during the pro- 
ject, additional resource allocations directly to science education at an 
institution. On the other hand, institutional financial support promised 
in the proposal might only be for indirect costs with no other resources 
forthcoming from the * stitution. We wondered if allocation of resources 
was a real indicator of institutional support, evidence for which is some- 
times difficult to track down. 

Continuation costs were also of forest to us because the program 
guidelines seemed to emphasize the need for institutions to continue support 
of a project once CAUSE fjunding was complete. We assumed this meant that 
many CAUSE projects were aimed at providing new and more expensive- to-operate 
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instructional resources. Therefore we chose to study the costs of project 
activities in terms of where the costs were incurred and for what purpose. 
Such an analysis we thought might provide information on recurring and 
non-recurring costs and their relationship to project continuation costs. 
A functional cost analysis of-project budget, activities, and objectives 
was selected as a way to describe the distribution of costs across pro- 
jects. We have studied three phases of a project which include: the 
design phase, when planning and development of project activities take 
place; the investment phase, when hardware and commercially-produced 
instructional materials are purchased or facilities are renovated; the 
operational phase, when the development c ;tivities are complete and any 
expenses are for maintaining use. Our goal has been to describe the 
% nature of costs within a variety of types of projects. 

Issue Five was selected as an important area for study for obvious 
reasons, but it was also a highly speculative area for study. The cost 
analysis of the kind of instructional improvement activity funded by 
CAUSE is a relatively new and difficult aspect of program evaluation 
endeavors. Furthermore, with projects and institutions as diverse as 
those funded by CAUSE it can be particularly difficult to avoid misleading, 
invalid or overly simplistic examinations of project costs. We therefore 
decided to limit the conduct of cost analysis to the eight sites chosen 
as part of the narrow focus evaluation, and to conduct each cost analysis 
with a primary concern for the accurate description of an individual 
project's utilization of its resources arid less of a concern for cross- 
project comparisons. As a result of these design decisions, relatively 
less data is available to address Issue Five than for the other issues, 
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but a concern for project cost variables ultimately figured in our under- 
standing of aspects of Issues Two and Three concerns. 

Policy recommendations . We viewed policy recommendations not so much 
as a separate issue, but as the overriding concern of this evaluation. 
We selected it as Issue Six but, in fact, we did not treat it as an issue 
to be investigated at each site. Our focus has been that the data for 
this issue would come from the other five issues rather than from any kind 
of direct data collection on policy recommendations. In this sense the 
issue of policy recommendations is a second order issue of the evaluation. 

Use of the Issues 

The issues have remained the same throughout the evaluation. At one 
point after the conduct of the first phase of the survey and the first 
visits to projects we seriously considered whether or not they would 
continue to be useful. At that time many new issues were emerging which 
did not seem to fit in topic or in scope with either our issues or our 
sub-issues. We decided without the issues the evaluation would lose 
all structure and coherence so we recorded and discussed the emerging 
topics. We then put them aside and continued to use the issues to focus 
our data collection and analysis activities.. 

As the evaluation progressed and we began analysis of data from 
each source we returned to the issues and the emergent topics. The 
topics, for us, had merged into the issues. They were important and 
visable but had found a space under the umbrella of one or another of the 
issues. In the meantime the sub-issues had changed. Some merged together. 
Others moved from the issue of quality of instruction to project implemen- 
tation or to the issue of institutional need, Several were changed to 
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better reflect the CAUSE program— the focus was similar but the perspec- 
tive had changed. 

Specific tracking of sub-issues across the span of the evaluation is 
difficult or impossible to do. For us the sub-issues served to bring us 
back to the specific evidence and detail needed to clarify each issue. 
We did not seek to maintain the sub-issues themselves as categories for 
findings and conclusions. A careful reader, however, should be able to 
see some influence of the sub-issues listed above in the discussions of 
findings in Volumes II and III. The six issues are the same. 



Evaluation Framework 



In order to conduct an evaluation based on issues we needed to 

select a framework to structure the evaluation itself. Stufflebeam 

describes the use of a framework in this way: 

An investigatory framework specifies the conditions 
under which data are to be gathered and reported, and 
the assumptions to be made in interpreting the findings. 
In all evaluation studies evaluators must choose either 
implicitly or explicitly among a number of alternative 
investigatory frameworks, e.g., experimental design, 
survey, case study, and site visitation. 

No one investigatory framework is superior in all 
cases. None is always best in serving the criteria 
of technical adequacy, utility, and efficiency. Also 
different frameworks work differentially well under 
different sets of feasibility constraints. Thus 
evaluators may choose different investigatory 
frameworks depending on the evaluative purposes to 
be served, the priorities assigned to the different 
criteria for judging evaluation reports, and the unique 
conditions under which evaluations are to be conducted. 
The task is to choose the framework that will optimize 
the quality and use of results under realistic constraints. 
(Stufflebeam, 1975, p. 47.) 
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Criteria for Selection of an Evaluation Framework 

Our overriding concern was to provide NSF with a useful evaluation 
of a complex, multidimensional funding program. Our professional bias 
was for the evaluation to take a formative role with respect to NSF policy- 
making. In selecting a framework we specified four criteria to guide the 
selection process: 

1. The methods should be capable of providing both a broad 
picture of the overall CAUSE program and in-depth portrayals 

i of a sample of individual projects. 

2. The methods should emphasize both redundancy and multiple 
perspectives. 

3. The methods should be capable of refining the evaluation 
as it proceeds. 

4. The methods should produce data that are policy-relevant and 
capable of being used by policy-makers. 

Each of these criteria is explained below. t 

Broad picture and in-depth portrayal . Cronbach (1970) has discussed 
what he calls the "bandwidth-fidelity dilemma" in terms of psychological 
testing. The dilemma is that there is always a trade-off between breadth 
of coverage and precision of information; i.e., it is possible in a given 
investigation to obtain a wealth of information about a few things, or a 
paucity of information about many things. An analogy may be drawn to the 
microscope. The highest powered lens will provide great detail but a 
narrow field of vision. The lowest powered lens will provide a wider 
field of vision, but some precision will be lost. This evaluation was 
designed to provide a comprehensive view of the CAUSE program through a 
combination of broad coverage/low precision techniques to provide an over- 
all picture of the program, and narrow coverage/high precision techniques 
to provide a detailed understanding of its more important facets. 
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Redundancy and multiple perspectives . Redundancy refers to the 
rep ated gathering of information from similar sources. This technique 
is used in physical, social and psychological measurement to increase 
reliability and to check validity. Multiple perspectives refer to the 
varied points of view of different observers. As in the story of the 
blind men and the elephant, reliance on the perceptions of one or two 
persons, whether they be site visitors or project staff, is likely to 
provide a distorted view of the CAUSE program. We chose data collection 
strategies which would maximize redundancy and minimize distortion caused 
by a limited number of perspectives. 

Refinement during the evaluation . This evaluation was designed to be 

a responsive evaluation in that the issues which were investigated were 

not limited to those identified in the planning phase. Stake's (1973) 

description of the portrayal process illustrates this principle: 

To do a responsive evaluation, the evaluator 
conceives of a plan of observations and negotiations. 
With the client's help he prepares brief narratives, 
portrayals, product displays, graphs, etc. He finds 
out what is of value to his audiences. He gathers 
expressions of worth from various individuals whose 
points of view differ. Of course, he checks the 
quality of his records. He gets program personnel 
to react to the accuracy of his portrayals. He gets 
authority figures to react to the importance of 
various findings. He does much of this informally- 
iterating, keeping a record of action and reaction. 

We chose methods which would permit an iterative process of issue definition 

and redefinition, based on imput from both NSF and individual projects. 

Policy-relevant data . EvaluatiCii data are notoriously under-utilized. 

Some of the reasons for the neglect of policy studies, according to 

Agarwala-Rogers (1977), include "lack of administrators' participation 

and involvement in the evaluation process, conflicting interests of program 
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officials and evaluators of the program, lack of mutually agreed upon 
'problem' definitions and 'needs' delineation between evaluators and users 
of the research findings . . ., lack of emphasis on providing solutions 
to problems . . ., overemphasis on negative aspects of the program . . ., 
and problems of feedback and timeliness of evaluation results." 

We chose a framework to involve NSF administration in the selection 
and definition of issues, to maximize NSF-DEA communication, and to focus 
on program successes as well as failures. These strategies were intended 
to maximize the utility of this evaluation for the development of NSF 
pol icy. 

Methods for the Framework 

To meet the criteria for selecting a framework for this evaluation 
we chose methods at three levels of specificity and focus. The first 
perspective, gained from case study methods, is a close-up and in-depth 
view of 8 out of 25 selected projects. The second perspective, from site 
visits, is a detailed examination of the remaining 17 of those 25 projects. 
The third perspective, gained from surveys of project directors and 
analysis of proposals, provides a broad view of all projects from 
1976-78 (N=201). 

■ The three perspectives were selected as ways to emphasize redundancy 
in data rol lection from multiple perspectives in order to increase relia- 
bility and validity of the. data. The issues provided the structure by 
which multiple perspectives could be investigated. Emphasis on different 
characteristics of the CAUSE program, at the program level and at the 
project level, could e be shifted among the three levels. With three levels 
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of methods to study a multidimensional program* the issues would have to 
be investigated in a way that would provide redundancy in the data. Other 
wise there would be no way to understand the relationship among levels or 
among the characteristics and dimensions of the CAUSE program. 

Accuracy of data can be examined by asking the same questions of 
different types of people and different sources. Redundancy could be 
achieved through use of complementary data sources and collection tech- 
niques. These range from the survey of project directors 1 opinions and 
experiences to the site visitors 1 observations over time of one site, 
from pre-planned interviews with project staff to informal conversations 
with students, and from investigation of specific sub-issued to explora- 
tion of unanticipated outcomes. The pursuit of multiple perspectives was 
chosen to lead to a more complete understanding of the issues. 

Strengths and Weaknesses of Methods 

Each data collection method had some strengths and weaknesses in its 
contribution to the evaluation. We chose methods to complement each other 
and to match the intent of the framework to provide redundancy and mul- 
tiple perspectives. For example, the survey asked a wide range of ques- 
tions of all project directors. Since the survey was conducted by mail 
and required written responses, the choice of questions was limited to 
those whose meaning could be conveyed to all project directors in written 
form. No opportunity existed to probe individual responses to clarify or 
extend their meaning. Validity and reliability of results was further 
affected by the fact that responses are based on self-report. The super- 
ficiality of the survey on some issues was balanced by the detail and 
depth of data that was provided by the case studies. Through the case 
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studies, team members had the opportunity to ask the same question of 
several CAUSE faculty and staff members and to repeat the question during ( 
return visits. However, the case study data are limited by the fact that 
they were collected by a'team of people who, because they visited a project 
several times, may have lost some of their perspective or objectivity 
with respect to a project. The two-day site visit, on the other hand, 
afforded teams the opportunity to ask similar questions from a wide 
variety of projects. The questions covered the same issues at each site 
but were tailored to each specific project. However, because these pro- 
jects were only visited once, the data were not as reliable as those 
modified by repeated visits and repeated questioning. The sites, as a 
.roup, add more breadth to the data than do the case studies and more 
detail than the broad view methods. Strengtns and weaknesses of the 
methods are summarized in Table 10. 

Data Collection Activities 

Visits to 25 CAUSE Projects 

The following section describes the data collection activities of 
the evaluation with respect to the narrow and medium focus methods. Since 
the 25 sites were selected as one sample of 1976-78 projects with the 
eight case study sites as a subset of the larger sample, site selection is 
discussed first. Following the discussion, site visits, case studies, and 
cost analysis are described. 

Site selection . Site selection was a complicated and difficult 
decision to make. The problem was <ius to the great diversity in CAUSE 
project and institutional characteristics, limited resources for site 
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Table 10 

Strengths and Weaknesses of Data Collection Methods 



Perspective, technique 
method and sample 



Strengths 



Weaknesses 



Broad focus: 
Mailed survey 

and 
Analysis of 
proposals 
N = 20ia 



Medium focus: 
Two-day site visit, 
including inter- 
views, observa- 
sation, and 
materials and 
document review 
N = 17 



Narrow focus: 
Case study 
including multiple 
site visits, em- 
phasis on progress 
of project, in- 
depth examination 
of some issues, 
and cost analysis 
N = 8 



Broad coverage; requires 
less time; Inexpensive, 
substantial redundancy 
across projects; oppor- 
tunity to investigate 
feasibility of program- 
wide baseline data col- 
lection; opportunity for 
all directors to air 
opinions; proposal analy- 
sis provides a comprehen- 
sive summary of the public 
face of CAUSE 

Opportunity to probe 
face- to- face on selected 
issues; to understand 
the project in its con- 
text; for visitors to 
interact with project 
staff on a professional 
level; cross-validation 
of subjective judgments 
from multiple perspec- 
tives; a significant 
sampling of project 
population 

Opportunity to understand 
issues in the context of 
an ongoing project; to 
look at the progress of 
a project over time; po- 
tential for establishing 
the rapport necessary for 
subtleties to be under- 
stood; multiple perspec- 
tives with substantial re- 
dundancy within sites; op- 
portunity to study expen- 
diture of project resources 



Relative superficiality of 
data collection techniques; 
inability to probe deeply; 
surveys limited to the 
single perspective of 
project directors; possi- 
bility of being perceived 
as bureaucratic meddling 



Possibility of visitors being 
"snowed" on a single visit; 
danger of over-generalization 
from single visits; lack of 
time to probe deeply into 
all important issues; pos- 
sibility of missing some 
important projects; reliance 
on subjectivity of visitors; 
limited redundancy 



Difficulty of generalization 
from small sample; substan- 
tial cooperation from sites 
necessary; time consuming; 
expensive; educational cost 
analysis is a relatively new 
technique and not well under- 
stood or well developed 



Because of the timelines of this evaluation, 1976, 1977 and 1978 projects 
were surveyed (N = 201). Proposals were analyzed for the years 1976, 1977, 



1978 and 1979 
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visiting, and the importance of representatives of the sample of sites. 
Diversity of projects is very great across variables such as the year 
funding began, duration of the project, type and size of institution, 
amount of funding, discipline focus, etc. Initial understanding of the 
importance of certain variables was confused by the brevity of project 
proposals and the complexity of project design. 

We selected the most important variables to consider and ranked them 
in terms of priority. We then selected a sample that was representative 
of the 201 projects from 1976-78. Because we were concerned about a 
number of different variables each had to be considered independently of 
the others. The sample of 25 represents several different distributions 
of variables. If the variables are considered simultaneously, the 
resulting sampling matrix has a great many cells, too many cells to be 
used as a guide to site selection. 

We distributed sites across two variables first: type of activity 
and funding level (see Table 11). We then checked the distribution of 
the other five variables across the sample. When we found an imbalance 
in some variable we "fiddled 11 with the sample by replacing some cases in 
order to even out the distribution. Eventually, we ended up with a fairly 
representative distribution of sites on each variable taken individually. 
The distribution across other variables is shown in Table 12. 

The variables w£ used in order of high to low priority are: type of 
activity, funding level, institutional involvement, year, institution 
type, discipline, and duration. "Type of activity" refers to the sub- 
stantive nature of the project and has six categories: use of computers, 
equipment and facilities, instructional materials revision, individualized 

i6 
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Table 11 

Distribution of the Sample (Population) Funding Level 
by Type of Activity 



Types of Activity 
Funding Use of Equipment/ Materials Individualized Remedi- 
Level Computers Facilities Revision Instruction ation Other 



Low Cost 
(less than 
$100,000) 


1 


(6) 


3 (21) 


1 (1) 


1 (5) 


1 (2) 


1 (ID 


Medium Cost 
($100,000 
to 150,000) 


2 


C.3) 


1 (15) 


0 (3) 


1 (6) 


0 (1) 


1 (6) 


High Cost 
(more than 
$150,000) 


3 


(33) 


2 (12) 


1 (5) 


3 (27) 


1 (ID 


2 (26) 


TOTAL 


6 


(49) 


6 (48) 


2 (9) 


5 (38) 


2 (14) 


4 (43) . 



Table 12 

Distribution of Sample Across Other Site Selection Variables 



I. Institutional Involvement: Single Department Across Departments 









28% 


72% 


II. 


Fiscal Funding Year: 


1976 


1977 


1978 




32% 


44% 


24% 


III. 


Type of Institution: 


2-yr. 


4-yr. 


Ph.D. Consortium 




28% 


40% 


24% 8% 


IV. 


Duration: 


1 year 


2 years 


3 years 






12% 


16% 


73% 



V. Disciplines: Life Science- (16%) Chemistry (12%) Earth Science (4%) 
Computer Science (4%) Engineering (8%) Physics (4%) 
Social Sciences (4%) Mathematic s (4%) Multi-discipline (44%) 
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instruction, remediation, and other. "Funding level" is a variable, based 
on NSF funding, which was created by dividing the range of awards from $0 
to $300,000 into three levels of $100,000 increments. "Institutional 
involvement' 1 is based on whether the project involves only one department 
at an institution or goes across departments. "Project year" is the 
fiscal year in which the project was awarded its grant: 1976, 1977, 1978. 
"Type of institution" is based on a distinction used throughout the CAUSE 
program of two-year colleges, baccalaureate degree-granting institutions 
(which may grant Masters degrees, too), Ph.D. -granting institutions, and 
consortia of institutions. "Discipline" is the academic field which is 
the focus of the project and again is taken from a designation used by 
NSF. The categories cover a broad range for each discipline and include: 
life sciences, chemistry, earth sciences, social sciences, physics, 
mathematics, computer science, engineering, and multi -discipline projects. 
Finally, "duration" of projects refers to their length as one year, two 
year or three year projects. 

One of the twenty-five sites we originally selected was not visited. 
The project had been scheduled to be completed during the period of our 
case studies. When we telephoned the college to speak to the project 
director we discovered that she had retired. We called her at home and 
found out that the project had been completed a year ahead of schedule. 
Although we intended to study some completed projects we decided that 
this particular project would be too difficult to study. We thought that 
with the project director gone from the institution the main source of 
information about the project was missing from the institution. We 
replaced the project in the sample with another (and it has been included 
in Tables 11 and 12). 
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Site visits . To prepare for site visits, consultants and staff met 
for a two-day orientation session. The purpose of the meeting was to 
explain CAUSE, the overall evaluation, the place of the, site visits in 
the evaluation, and site visit procedures. We tried to build a shared 
understanding of the evaluation issues and the goals of the evaluation. 
Together we reviewed and discussed site visit procedures as described in' 
a site visit resource manual and an individual site visit guide. 

The site visit resource manual contained: the names and addresses 
of the evaluation staff und consultants; a summary of the evaluation. plan; 
some information on interviewing, materials review, and cost analysis; a 
statement of confidentiality of site visit data; a CAUSE program solici- 
tation, statistics on CAUSE projects, 1976-78; and guidelines for travel 
expenses. The manual provided background material for the individual 
site visit guide. A separate copy of the guide was prepared for every 
site visit. It was taken on a site visit by both team members and used 
as a tool for preparing for a visit and as a place to record data. The 
individual site visit guide contained a checklist for all procedures; a 
schedule for the visit; interview planning guides; an interviewing focusing 
guide to the evaluation series; a form for recording the focus of the visit 
a form for planning each interview; de-briefing procedures for between 
team members; and specific directions for report preparation. Forms for 
giving feedback on the visit to DEA were included and were completed by 
the project director and by the science educator. 

The site visits were conducted by two-person teams, composed of a 
science educator and an evaluator, in order to represent two points of 
view. The viewpoints were unique in some respects and overlapping in 
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others. The science educator represented discipline expertise and know- 
ledge of current practices in science teaching. The educational evaluator 
represented expertise in current instructional improvement practices in 
higher education and, as a full-time DEA staff member, consistency with 
the rest of the evaluation activities. Science educators were assigned 
to sites based on their discipline and their expertise. 

Activities related to a typical site visit began two or three months 
in advance of the visit. Project directors were notified of our interest 
in having them participate in the evaluation and a time was set for the 
visit. The project director was asked to set up a two-day schedule of 
interviews, observation and group meetings. The science educator and the 
evaluator read the project proposal, discussed it, selected a focus for 
the visit, and wrote questions on issues via telephone in advance of the 
visit. 

The site visitors met on site the morning of the first day. They 
made final plans for the visit before going to the institution. In 
advance of the teams 1 arrival project directors set up group meetings and/ 
or individual interviews with CAUSE faculty and staff, students, non-CAUSE 
science faculty, and administrators. The team usually met first with the 
project 'director to discuss their schedule and to make additions or 
changes. 

Each day on site included extensive interviewing, touring of facili- 
ties, observing classes and reviewing materials and project documents. 
At the end of the first day the two team members went over their observa- 
tions, discussed them, and selected questions which had been missed or 
were still unclear. At the end of the second day they met with the pro- 
ject director for a debriefing session. 
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Reports of the visit were the responsibility of both team members. 
The science educator followed a specified format which covered a descrip- 
tion of the site visit and an analysis of the issues with respect to the 
project. The evaluator wrote informal field notes summarizing each acti- 
vity of the site visit and then later rewrote the science educators 
report. Then the report was sent to the project director for corrections 
and consents. Based on the comments, the site visit reports were revised. 
In a second round of revisions the site visits were rewritten to include 
more information on the institution, the techniques used for the site 
visit, a better description of the project, and fictional names of people 
and institutions. 

Case studies . The data collection activities of the case studies 
were essentially the same as those of the site visits. What differed was 
the addition of multiple observations over the period of a year. The 
issues were explored in more depth, specifically in respect to how the 
project progressed and evolved over time. Between visits the team 
members and other DEA evaluation staff read the reports, commented on 
them, and identified topics to examine in more detail. The team had time 
to obtain and to study project documents and instructional materials 
carefully. 

Reports were written, in most cases, after the first visit and after 
the last visit. The interim visits were documented with field notes. 
The first reports were sent to project directors for comment and, 
generally, elicited many corrections. 

The relationship between the visiting team and the project, especially 
the project director, was different from that of the site visits, and it 
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altered over time. The relationship differed because the projects knew 
the teams would be visiting again. It might be described as a tension 
which existed between wanting to be friendly and hospitable and feeling 
threatened by being evaluated. In all cases, however, project directors 
and staff were very cordial and cooperative. 

Reporting format for case studies was not specified as rigidly as it 
was for the site visits. Teams were given more leeway to design a report 
which described the project and the casing process as they decided best 
fit the situation. The focus of the case studies was always on the 
evaluation issues. However, at each institution themes, or additional 
issues, developed as the team investigated further. The reports, in 
^.■^vYjI, and the prefaces and summaries of the reports, in specific, are 
intended to describe the special themes of a project in addition to the 
issues. 

Cost analysis . Each case study has a component which explored pro- 
ject posts. A cost analyst visited the case study site and gathered data 
independent of the rest of the case study team's data collection activi- 
ties. The analysts were kept abreast of the results of visits to the 
sites, and no analyst went to a site until the case study team had visited 
at least once. The analyst interviewed project staff to gather data about 
the expenditure of time, money and other resources which were used to 
accomplish the objectives of the project. 

Typically, total project costs were first analyzed by conventional 
budget categories for an established fiscal period and displayed on a 
standard budget form. This chronological compilation of costs by cate- 
gories did not show information useful in understanding some project 
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characteristics.. For example, traditional budget categories by fiscal 
year did hot show the distribution of costs for instructional improvement 
projects which have three distinct phases of activity: design, invest- 
ment, and operation. Thus, a second, functional analysis of project costs 
was conducted which arranged project costs according to project objectives 
or activities. While the analysis based on traditional budget categories 
was useful in examining continuation costs, the functional analysis was 
useful in examining the nature, sources, and allocation of contributed 
resources (particularly those not listed in the original proposal). 

The cost analysis methods were not utilized in any uniform manner 
across the case study sites. The objectives of the projects varied 
greatly and the sample is small. So the cost analysts chose, as a team, 
to select methods for each case study site which would best illustrate 
the type of costs for that project. Further discussion of the selection 
of methods is continued in the section in Chapter One, Volume II which 
summarizes findings on project costs. 

Broad View of CAUSE Projects 

Survey of CAUSE pro.ie ct directors . A survey of virtually all project 
directors for 1976, 1977, and 1978 projects was conducted in two stages. 
One-half of the' project directors received the first survey and the other 
half received the second survey. The survey results provide a broad 
overview of CAUSE projects from the perspective of the project directors. 
They reflect the project directors' experience with project activities 
and their opinions about their projects and the CAUSE program. A complete 
description of the rationale, procedures, and results from the survey appear 
in Volume III, Chapter One. 
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Analysis of proposals . A content analysis of proposals for 1976-79 

f 

CAUSE projects was conducted as a second broad focus data collection 
activity. The original intention of the content analysis was to examine 
science education and institutional needs described in proposals. The 
analysis was to provide an overall description of the various types of 
science education needs being served by CAUSE and the relative emphasis 
the needs were given in the proposals. The analysis was expanded to 
include an examination of problems and needs, audiences for CAUSE projects, 
goals and objectives, and outcomes of projects as proposed. The additional 

categories fitted easily into the content analysis data collection and 
parallelled an earlier analysis conducted by NSF. 

What did not work which was proposed was analysis of unfunded 
proposals. We took a sample of 20 unfunded proposals from 1978. The 
sample was too small to be meaningful for comparison purposes to funded 
proposals for 1978. There were too many "empty cells" or categories 
of variables which did not have any cases. We dropped the unfunded 
proposals from further comparison. A complete report of the analysis of 
proposals including procedures, data analysis, and results appears in 
Volume III, Chapter Two. 

Data Analysis 

Data analysis was accomplished through use of an iterative process 
of discovery and verification. The discovery phase was the expansion of 
the sample of observations using the methods of site visiting, casing, 
surveying, and content analysis. The verification phase was one of 
reduction of observations into categories and relationships among 
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categories. We used this two-phase process over and over again during 
the evaluation. 

The derivation of the six issues during the planning stage of the 
evaluation was the first round of iterations and is explained above in 
the section on the issues. Once the evaluation was underway, observations 
were gathered from the first survey, the analysis of proposals, and about 
one-fourth of the visits to projects. The evaluation team then got 
together to analyze the first group of findings. Since these observations 
had been gathered with instruments and interview protocols based on the 
six issues and the sub-issues, they were easily organized into categories 
which matched the issues. However, we had many findings which did not 
fit. We debated elimination of the issues or the sub-issues. We tried 
to explain the meaning of the uncategorized findings. We proposed new 
categories but those could not incorporate all the findings, just the 
ones which were outside the issues. Eventually we decided on some rather 
weak new categories and planned to investigate them as well as the issues 
in the next round of data collection. 

After completion of the second survey and about two-thirds of the 
visits we met again to analyze findings. The new categories proposed in 
the previous interaction had lost their ability to explain findings in a 
meaningful way. The issues were useful in explaining all the old and 
the new observations taken together but many of the sub-issues did not 
work too well . 

At the end of all of the data collection, findings were analyzed 
again. We worked with the science education consultants to select 
categories to explain and reduce the observations. The conclusions 
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(Volume I, Chapter Three), the analysis of findings from the case studies 
and site visits (Volume II, Chapter One) and the results of the survey and 
the content analysis (Volume III) represent the outcome of our final 
round of analyses. The issues remained viable throughout the evaluation 
as categories for reducing the data. The sub-issues were less useful and 
new categories or sub-issues were generally employed in analyzing the 
results as treated in the respective chapters. 

The validity of the issues and the categories of findings under each 
issue can be tested by examining them with respect to the data reported 
in the site visit reports, the case studies, and the tables from the 
survey and the analysis of proposals. The issues and categories should 
explain the meaning of the findings :nd should not be able to be contra- 
dicted. No counter-examples to the categories should exist- if oxir analy- 
sis was adequate. The analyses and conclusions based on the issues should 
bear a general izable relevance across the data as reported here. 



The methodology of the evaluation has been built across six issues: 
institutional need, project implementation, quality of instruction, 
evaluation, project costs, and recommendations. The evaluation framework 
emphasized redundancy and multiple perspectives. The methods were 
selected at three levels of focus: broad, medium, and narrow. Site 
visits, case studies- surveys, and content analysis were chosen as data 
collection techniques. The data analysis process was a naturalistic one 
which utilized the iterative process of discovery and verification. 



Summary 
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Results were analyzed and reported by level of focus: The findings from 
medium and narrow- focused efforts are presented in Chapter One of 
Volume II; Chapter One of Volume III reports the outcomes of the broad- 
focused investigations; and, the remaining chapters of this volume report 
conclusions and recommendations based upon all three levels. 
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CHAPTER THREE 
CONCLUSIONS 

Introduction 

n This chapter presents the evaluation's overall conclusions. The 
conclusions are organized around the issues whose choice and definition 
is explained in the preceding chapter. Various conclusions are presented 
relevant to each issue. An explanation of each conclusion is provided 
along with a series of references to some of the specific evaluation 
findings which led to the conclusion. The reader is encouraged to use 
these references as a bridge to the more detailed findings and evidence 
presented in 'Volumes II and III. 

The contents of this chapter represent the ultimate level of refine- 
ment for the evidence gathered through the evaluation's various activi- 
ties. Other analyses of findings are presented in the first chapters of 
Volumes II and III, but these analyses are solely at the level of the 
study's narrow and medium, or broad focused methods respectively. The 
conclusions presented in this chapter are based on all aspects of the 
evaluation. Chapter Four, then, go_s one step beyond findings themselves 
to offer recommendations for the future conduct of the CAUSE program. 

Finally, before considering the conclusions it should be noted that 
all projects selected for site visits and case studies have been treated 
anonymously in this report. Fictitious names have been used for every 
institution and person involved in the study. 
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Issue One 

To What Extent Are High Priority Institutional 
Needs Being Met by Local CAUSE Projects? 

Conclusion 1.1: Most CAUSE Projects Would Not Have Occurred Without 
CAUSE funding . 

This is perhaps the most basic conclusion of the evaluation. It 
confirms the fact that the NSF funds are necessary to establish projects 
of the type sponsored by CAUSE. (This, of course, implies nothing about 
the importance or appropriateness of the projects themselves, which will 
be dealt with in later conclusions.) 

Approximately half of the project directors surveyed and visited 
reported that they would have "given up on their projects for lack of 
funds" if CAUSE money had not been available. The reports of the site 
visitors confirm this perception. It appears, further, that those who 
would have proceeded without CAUSE funds would have had to do so at an 
extremely reduced capacity and in almost all cases with a drastically 
different project. 

The major reason for this is that most universities are facing or 
will soon face decreasing enrollments and are operating under fixed or 
shrinking budgets. A project funded internally must be viewed in terms 
of other internal functions that must be cut back for its support. There 
are few loose dollars anywhere in higher education. 

A related reason is that most of the projects funded by CAUSE require 
an up-front investment of relatively large amounts of dollars. Facilities 
and equipment for science on a per student basis are expensive. When 
money is available for new resources science must compete with other 
departments and science may not be a high priority of the institution. 
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The need for relatively large amounts of dollars is in part due to the 
need to acquire or renovate facilities or equipment, but it is also due 
to the comprehensive nature of CAUSE projects. Most projects which 
require the cooperation of faculty and administration have to represent 
a wide variation of interests and concerns. The resources necessary to 
coordinate and integrate the various aspects of the projects with related 
components are substantial compared to normal day-to-day activities. 
Moreover, in order to obtain a unified effort, it is necessary to involve 
and coordinate the expenditure of these resources fairly simultaneously. 
Otherwise, the project runs a strong risk of being lost in the shuffle. 

Finally, another reason for believing that the CAUSE projects we 
reviewed would not have occurred without external support is that there 
often is an uncertain commitment to the improvement of instruction in 
science education, particularly in terms of curriculum revisions and 
additions, and faculty development. Just over half of the project direc- 
tors reported that no incentives were provided by their institution to 
work on CAUSE. Visitors to sites often had the impression that project 
staff were fighting an uphill battle to maintain institutional commitment 
from fellow faculty as well as from the administration for the improvement 
of instruction. Faculty who devote their available professional time to 
research are rewarded with promotion, tenure and prestige. Those who 
choose to concentrate their efforts on instruction are often locked down 
upon by their colleagues and rarely receive any sort of credit toward 
promotion and tenure. We observed a few instances in which faculty 
involvement in instructional improvement may have hindered promotion. 

The lack of available internal funds as well as the lack of strong 
institutional commitment for instructional improvement led us to believe 
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that CAUSE funding was necessary for these projects to have come into 
existence. 

Example references to the evaluation findings . The following points 
represent some of the specific observations which gave rise to this con- 
clusion. These points are offered to help the reader understand how this 
conclusion was reached. For a more complete understanding, the reader is 
encouraged to refer to the material in Volumes II and III. 

1. Forty-six percent of project directors reported that they would 
have given up their projects without CAUSE funding. Most of the 
rest identified other sources of external funding. Only 6% 
reported that they would have sought funding from within their 
institution. 

2. Over one-half of the project directors surveyed reported that 
they received no incentives for working on CAUSE. Of those, 
over one-half reported that administrative recognition and 
release time would have been helpful in achieving project goals. 

3. Maples CCC and Saints University are good examples of projects 
that could not have occurred at all without CAUSE funds. Sands 
College* Elms College, and College of the Mountains provide 
examples of projects that probably would have occurred without 
CAUSE funds, but at a much lower level or over a much longer 
period of time. 

4. Although shortages of funds are a problem everywhere, Springs 
University and Forestview College provide clear pictures of the 
role CAUSE projects can play in institutions where budgets are 
restricted. 

5. Cedar State University illustrates the need for intensive front- 
end efforts to bring about a successful project; Sands College, 
Elms College, and Willows University illustrate the need for 
front-end equipment and facilities purchases. 

6. Ivy University and University of the River provide examples of 
the tension which can arise between the research and instruction 
responsibilities of faculty at some institutions. 



Conclusion 1.2: CAUSE Projects Fill Needs Perceived as Important within 
Institutions which Receive the Grants . 

The CAUSE program stresses a focus on locally defined needs as 

opposed to more general, nationally defined needs. The hope expressed 
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in the legislation is that the CAUSE program '\ , , will permit depart- 
ments and institutions to develop their own approaches to science teaching. 1 
The CAUSE program guidelines require a statement from the institutional 
administration (usually signed by the president or academic vice-president) 
describing the project's relationship to institutional goals and stating 
the institution's commitment to and support of the project's implementa- 
tion and post-grant continuation. This description and justification of 
local needs is seen as one means of insuring the necessary institutional 
support for project implementation and the eventual continuation of pro- 
ject improvements in post-grant years. 

In and of itself, the fact that a given institution took the efforts 
necessary to pursue a CAUSE project suggests at least some level of con- 
cern for the need(s) addressed by the project. Evidence gathered in the 
conduct of this study indicates that this assumption is an appropriate 
one. The more important question, however, is whether there is a broad- 
based perception of the importance of the need(s) throughout the institu- 
tion. This question is a more difficult one to answer. 

Primarily from the evidence collected as part of the site visit and 
case study methods of this evaluation, it is our judgment that the needs 
served by CAUSE projects are generally of high priority for the institu- 
tion. In a few cases, the need being addressed was of crucial importance 
to the successful maintenance of any kind of viable science education 
program. In a few cases, it appeared that the needs addressed by the 
projects were of only tangential importance to the institutions. Most 
projects addressed needs which fell somewhere in between these extremes. 
From both visits to the sites and the survey data, two important 
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variables relating to the relationship between CAUSE projects and insti- 
tutional needs have been identified. First, there sometimes was reason 
to challenge the assumption that simply because the administration of the 
institution "signs off" on a proposal there is broad based agreement that 
the needs addressed are of high priority. In some cases it might have 
been that some members of the institution simply saw the project as a 
means of obtaining external dollars; in other cases, it might have been 
that the institutions saw the primary benefit of the CAUSE project as an 
increase in institutional prestige. In short, the reasons for local 
support of a CAUSE project are not always as simple as they appear in the 
proposal. 

A second variable affecting the relationship between CAUSE projects 
and institutional needs involves the intra-insti tutional resolution of 
competing plans for CAUSE projects. CAUSE guidelines specify that only 
one project within an institution can be funded during a given time period 
Evidence collected suggests that where there are competing proposals withi 
an institution, decisions about which one the institution will commit 
itself to are not always made totally on the basis of the most important 
institutional needs. Instead, these decisions are sometimes made on the 
basis of the ability of various departments or divisions to handle insti- 
tutional politics, or on the basis of which proposal has the highest 
probability of being funded. Increased attention to the ways in which 
institutional needs are identified and the relative priority of these 
needs appears warranted. 

Example references to the evaluation findings . The following points 
represent some of the evidence which gave rise to this conclusion. 
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1. Surveyed project directors reported in large numbers that such 
activities as "efforts to win support for our project at our 
institution" were important for project success, while "reluc- 
tance of important department or school administrators to commit 
themselves to the project" and "communication problems within 
our institution" were two of the most frequently cited diffi- 
culties. 

2. An outcome noted by several project directors on the survey was 
that the CAUSE project sometimes acted as a catalyst to other 
departments or units on campus to initiate activities similar 
to those carried out on the CAUSE project, 

3. Sands College and Saints University have CAUSE projects which 
have a very broad base of support within their institutions. 

4. The project at Ivy University may face some difficulties because 
of a lack of a broad base of support within the institution. 

5. At College of the Mountains, the chemistry department's CAUSE 
proposal was chosen for support by the administration over 
biology's in part because chemistry's proposal had a higher 
likelihood of success. 



Issue Two 

How Are the CAUSE Projects Being Implemented? 
What Are Their Strengths and Weaknesses? 

Conclusion 2.1: CAUSE Projects are Being Implemented As Proposed . 

The relationship between the project as described in the proposal and 
the project as implemented generally was an initial focus of the site 
visitors, not as an assessment cf compliance but as a means of becoming 
quickly acquainted with the nature of the ongoing project. The survey of 
project directors also asked several questions on this point. In the 
opinion of the project directors as well as^the site visitors, there was 
a very close match between projects as proposed and projects as imple- 
mented. 

This generalization, however, should be qualified on two counts. 
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First, there frequently were some variations from original plans, usually 
in terms of emphasis and occasionally through the dropping of one or two 
original objectives. Sometimes this was done through a formal amendment 
to the grant, but more often it simply occurred during project imple- 
mentation. The variations were rarely significant in terms of the pro- 
ject's overall goals and often represented reasonable judgments on the 
part of the project director. The need for modifications usually resulted 
either from an initial lack of understanding of the resources necessary 
to carry out the proposed tasks (this was especially true with respect to 
materials and curriculum design activities) or from a mid-project decision 
to de-emphasize the project's relatively unsuccessful aspects and empha- 
size its more successful aspects. Occasionally changes had to be made 
due to changes in project personnel. 

A second qualification to the conclusion that projects are imple- 
mented as proposed relates to the clarity with which project plans and 
objectives were originally stated. Projects differed widely with respect 
to the specificity of goals and objectives; those with clearly specified 
goals and objectives were more easily assessed in terms of the degree to 
which they accomplished their original intents. Project directors in 
these projects also tended to monitor the progress of the project more 
closely than those in projects in which goals and objectives were defined 
ambiguously, possibly because the specification of goals and objectives 
made such monitoring possible. However, even those projects without 
specifically defined objectives appeared to conform to the spirit and 
general intent of their proposals. 

Finally, it should be noted that almost all projects had to make 
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changes in their original timelines. This was often due to delays in 
receiving equipment purchases and/or to unfamiliari ty with the amount of 
time activities and processes take. Plans for project expenditures were 
also frequently modified. This usually occurred formally through the 
institutions contributing more than was originally intended. Occasionally, 
formal renegotiations with NSF were required. Evaluation plans also were 
frequently modified for reasons to be discussed later. 

Example references to the evaluation findings . The following points 
are offered to help the reader understand how this conclusion was reached. 
For further information, the reader is encouraged to refer to material in 
Volumes II and III (particularly Volume II). 

1. The survey showed project directors reporting that goals and 
objectives, activities, management and impacts have been very 
much as proposed. Budget, evaluation plans and timelines are 
reported as frequently changed from the original proposal. 

2. University of the River's project is an example of a modifica- 
tion of one objective - purchasing some commerically-produced 
materials for the learning center instead of developing all 
materials locally - and a modification in timeline due to 
university scheduling. 

3. Saints University and Cedar State University are examples of 
projects which followed their objectives very closely as they 
were specified. 

4. Clay College and Valley University are examples of projects 
which had delayed timelines because of unexpected delays in 
start-up activities. 



Conclusion 2.2: CAUSE Projects Serve as Catalysts in Focusing, Motiva- 
ting and Coordinating Instructional Improvement Activities of Science 
Faculty . 

High quality projects are characterized by high quality project staff. 
While these people generally were present at the institution before the 
award of the grant, due to various institutional constraints, they may 
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not have worked together previously on a major instructional improvement 
effort. The CAUSE project acts as a catalyst and overcomes some of these 
real or perceived constraints and brings these people together. Typical 
constraints which CAUSE funding can help to overcome include lack of time, 
opportunity (and often precedent) to work with people other than immediate 
colleagues, and a general lack of substantive support for the improvement 
of instruction within the institution. 

The results of the survey of project directors indicate the impor- 
tance to the projects 1 success of a committed staff working collaboratively. 
Responses to these surveys and the case study data also indicate that the 
opportunity to collaborate with other science educators is an important 
benefit of participating in the project. It was evident at many of the 
site visits, particularly at the more successful projects, that faculty 
who had not previously worked collaboratively were now working as a team, 
often providing substantial encouragement and support to each other with 
the result that the team's combined efforts were often substantially 
greater than the sum of the individuals 1 efforts could ever have been. 

The collaborative nature of CAUSE projects provides benefits in 
addition to the support of colleagues. The presence of a single large 
project provides each participant with more prestige and recognition than 
would be provided faculty members if they performed the same activities 
on their own. The visibility of the projects also facilitates the 
mobilization of other institutional resources in its support. Most 
importantly, the collaborative and often interdisciplinary nature of CAUSE 
projects encourages the weaving of instructional improvements into the 
fabric of science programs at the institution. The coordination of an 
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institution 1 s already existing personnel resources into effective teams 
committed to instructional improvement is perhaps one of the CAUSE pro- 
gram's more important contributions to science education. 

Example references to the evaluation findings . The following points 
represent some of the specific observations which gave rise to this con- 
clusion. See Volumes II and III for further details. 

1. The survey results show an overwhelming majority of project 
directors reporting that "working collaboratively with project 
staff" is important to project success. 

2. The survey also suggests that improved morale and a sense of 
community are unexpected benefits of a well functioning CAU5E 
team. 

3. The negative effects of changes in staff and/or of unproductive 
staff members were also reported by project directors on the 
survey. 

4. Saints University's CAUSE faculty mentioned as a "best aspect" 
of the project the opportunity to work together with faculty 
from other departments to attack attrition problems in entry- 
level science courses. The project director there promoted 
dissemination of information about the CAUSE project and, 
thereby promoted recognition of individual faculty efforts. 

5. At Central City JC, several faculty mentioned that increased 
communications and collaboration among faculty from different 
science divisions was not only unexpected, but also one of the 
most important consequences of the project. 

6. Cedar State University provides an outstanding example of the 
benefits of a collaborative relationship among faculty. 



Conclusion 2.3: Projects Are Generally Adequately Managed, but Project 
Directors Often Do Not Have Access to Areas of Expertise Important to 
Project Implementation . 

While it was found that most projects are achieving (sometimes sur- 
passing^ their proposed objectives, it also is the case that these 
achievements are frequently limited by or require more effort than neces 
sary due to a lack of the required expertise. Generally, this lack of 
expertise is inert in the area of project management since project 
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directors tend to be experienced faculty who have the respect of their 
colleagues and are effective loaders. Instead, the lack of expertise 
generally involves some specific activity of t o project itself. Such 
deficiencies were often recognized in the proposal and plans were made 
to compensate for the missing expertise by hiring consultants. Some- 
times, however, the need for certain kinds of expertise had not teen 
adequately recognized in planning the project. 

The area in which expertise was most frequently lacking was instruc- 
tional development. Although most project directors and staff have had 
extensive previous experience in teaching and often in the design of 
their own instructional materials, few have had experience with (or 
sometimes recognize the need for) a systematic approach which stresses 
the clarification of needs to be served, the identification and choice of 
instructional options, the development of the selected options and the use 
of feedback information to improve their implementation. The evaluators 
felt that CAUSE dollars could have been more efficiently used in general 
had instructional design expertise been available. The lack of adequate 
evaluation expertise is a related problem to be discussed under the con- 
clusions related to Issue Four. Other types of expertise that were not 
always accessed when appropriate include hardware selection (particularly 
with respect to computers), computer programming, and the production of 
instructional television. 

In general, project directors seem to have a good grasp of general 
project management skills such as planning, budgeting, monitoring, com- 
munication and allocation of responsibility. However, some did not and 
their projects suffered accordingly. Timelines were often unrealistic 
and project directors and staff were frequently surprised by the amount 
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of time various tasks consumed. The time required for project management 
activities whe^selves was usually underbudgeted. 

An important and potentially valuable source of expertise that pro- 
ject directors expressed strong interest in was the group of present and 
previous CAUSE project directors themselves. When asked to make one sug- 
gestion to NSF in an open ended question on each of the surveys to pro- 
ject directors, 40% and 43% of the suggestions related to more and better 
communication and dissemination among CAUSE presets and with NSF staff. 
Based on the site visitors 1 observations, it is undoubtedly true that the 
staff of the various CAUSE projects could learn a lot from each other's 
experiences. 

Example references to the evaluation findings . The following points 
suggest some of the observations which gave rise to this conclusion. 
Volumes II and III provide further evidence, 

1. On the survey, the most frequently mentioned areas of expertise 
that project directors thought would have been helpful were 
evaluation, computer applications and project management. 
Interestingly, less often mentioned was instructional development. 

2. It seems likely that some of the computing modules developed on 
the CCHEI project could have been made more instructionaUy 
effective if an instructional designer had assisted faculty. 
Some faculty specifically mentioned difficulties they had with 
organizing the content for instructional purposes, 

3. The project director at Maples CCC had developed a comprehensive 
system for monitoring -j,- progress of a project which spans a 
multi-campus college. 

4. The projects at Willows University and Sea University provide 
examples of how persons with expertise in computer programming 
can be integrated into a project team. The project director at 
Cedar State University was adept at capitalizing on the tech- 
nical skills of others within the university. 



Conclusion 2,4: Effective Project Directors Tend to Possess Several 
Common Characteristics , 

The need for project directors who are able to effectively direct 
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and administer project activities is critical to the successful culmina- 
tion of CAUSE projects. Most project directors at sites visited appeared 
to be generally effective in their roles. Several appeared to be out- 
standing. Some of the characteristics which seem to be related to project 
director effectiveness are outlined below. 

In most cases, project directors who are effective are senior faculty 
members and had primary responsibility for writing the proposal. Their 
involvement in the proposal. writing meant that they were the ones pri- 
marily responsible for formulating project objectives and implementation 
plans. Thus they had thorough knowledge and understanding of the rationale 
of project activities from the inception of project implementation. In 
most successful projects which span departments or divisions of the insti- 
tution, the project directors hold administrative positions such as 
department chairperson. Presumably, senior faculty are more able than 
junior faculty to weather the political implications of administering 
across organizational units and are, in addition, less susceptible to 
"publish or perish 11 pressures. 

Good project directors provide strong personal leadership and are 
innovators at their institutions. Generally they are among the first at 
their institutions to urge adoption of local innovative activities, 
typically in the guise of a CAUSE project. Most are well respected among 
colleagues and administrators and set examples of hard work and enthusiasm 
for the project. A related characteristic is that effective project 
directors are frequently adept at handling institutional politico. This 
appears to be most important when projects cross organizational lines 
within the institution. 
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Some of trie most successful project directors appear to be those 
who are experienced at and knowledgeable about the relevant substantive 
areas and tasks of the project. There are several notable exceptions to 
this, however, where'project directors are adept at utilizing the exper- 
tise of others at critical project points. 

Example references to the evaluation findings . The following points 
are offered as examples of the evidence relevant to this conclusion, 

1. When we asked, on the survey, whether project directors had had 
previous experience in managing externally-funded projects in a 
higher education setting, 42% reported having managed at least 
one instructional improvement project prior to CAUSE; 39% 
reported having managed at least one research project prior to 
CAUSE, and 31% reported that CAUSE was their first experience 
in managing a project. 

2. Project directors at Maples CCC, Saints University, Central 
City JC, Willows University and Coastal University have strong 
personal leadership styles and are all quite adept at institu- 
tional politics. 

3. The project director at Cedar State University was extremely 
skilled at project management and also adept at capitalizing 
on the skills of others within the institution. 

4. The project director at Central City JC was adept at encouraging 
the active participation of administrators in project activities, 
thus promoting their ownership of project goals and their support. 

Conclusion 2 5: The Use of Release Time Is a Critical Factor in the 
Efficient and Effective Use of CAUSE Dollars . 

The use of release time appears to be of extreme importance in the 
successful completion of CAUSE projects. CAUSE projects represent work 
for faculty which is in addition to their normally assigned responsibili- 
ties. Thus, either the faculty member must be released from his/her usual 
responsibilities in order to devote time and effort to project activities 
or the faculty member must complete project tasks during his/her personal 
time in addition to normal responsibilities. Most CAUSE proposals specify 
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the use of release time for faculty to work on projects. In implementing 
the projects, however, it is sometimes very difficult to allocate or 
administer release tirce effectively. 

Problems involving the effective allocation of release time generally 
stem from a number of sources including the nature and size of the insti- 
tution, the accuracy of the planned-for level of effort, institutional 
policies, and the willingness of institutional administrators to live up 
to commitments made. The most effective utilization of release time was 
when faculty were directly released from a percentage of their teaching 
responsibilities. At some institutions, however, this proved to be 
difficult either because the size of the institution did not provide the 
flexibility necessary for other faculty to cover the course(s) from which 
an individual was released or because adequate replacement faculty could 
not be found. At some institutions, faculty were released from non- 
teaching responsibilities (such as committee assignments), but in most 
cases this turned out to be release time more in theory than in reality. 
In at least one case among the sites visited, the release time cororri tments 
made by the institution were simply ignored. Several projects allocated 
release time as overload pay (a dubious practice given NSF guidelines). 
For some projects, the timing of release time and the intensity of pro- 
ject activities proved to be a problem. 

In most cases it was found that whatever release time was made 
available was generally insufficient. Most project plans underestimated 
the time required for critical project tasks such as development of 
instructional materials. Almost all projects underestimated the time 
required for the project director to administer the project. 
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Problems associated with the insufficiency of release time resulted 
in several observed and reported negative consequences to project 
operation and to the potential for project continuance. Chief among these 
were the "burn out" of project faculty (most notably project directors) 
and the fact that project timelines in many cases needed to be extended 
because not enough "real time" was available to work on project tasks. 

Example references to the evaluation findings . The following points 
suggest some of the observations which led to this conclusion. A number 
of the case studies (and particularly the cost analyses) provide further 
detail on this area. 

1. The survey shows that "lack of sufficient time to complete 
planned activities"; and "conflicting coironitments on the part 
of project staff", two items added to the second version of the 
survey ? were perceived by project directors to be the most 
serious difficulties in their projects. The survey also shows 
that only 44% of the project directors reported that incentives 
for working on CAUSE were provided by their institution. 

2. The most extreme example of problems with release-time is CCHEI 
where none was initially built into the project implementation. 

3. Project directors at Saints University and University of the 
River all could have benefited from more release time. 

4. At Bay College release time which had originally been allocated 
never materialized for the project director or other project 
staff. 

5. At Cedar State, conflicting responsibilities for research and 
teaching made the effective use of release time difficult. 

Conclusion 2.6: Institutional Support Is Critical to Successful Project 
Implementation . 

Horizontal and vertical support for CAUSE projects appears to be 
generally adequate for project success. However, there does seem to be 
definite room for improvement, and for some projects, lack of support is 
a serious problem. This lack of support is sometimes due to a lack of 
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skills, abilities, and political power of the project director and pro- 
ject faculty. At other times it is due to a lack of a clear understanding 
and definition of what institutional needs are being served by the project 
and why they are of high priority. 

It is very important that sufficient institutional support be 
obtained prior to the submission of the proposal. In some cases, written 
documentation of such support has been used to hold administrators or 
faculty to earlier commitments once the project was funded. It is also 
critical that projects generate sufficient institutional support during 
project execution so that adequate vertical and horizontal support will 
continue to exist for project activities in the post-grant era. Those 
projects which have never gained either the support of faculty or 
administrators at their institutions tend to have a lower probability of 
successful continuation. 

Faculty reward structures are related to the notion of institutional 
support. In addition to providing project staff with adequate time to 
accomplish the tasks of the projects, it is also important that adequate 
reward structures for faculty participation in the project be provided. 
This can be an especially severe problem in larger institutions with 
strong research orientations. Typically, time and effort spent on 
instructional improvement projects do little for the faculty member when 
questions of promotion and tenure arise. This problem seems related to 
the fact that most successful project directors are senior faculty members 
who are substantially insulated from such pressures. In many cases, the 
only apparent reward available to faculty who work on the project is the 
satisfaction of knowing that the facilities and/or resources for Science 
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instruction at their institution have been improved and presumably that 
science instruction itself has been improved. 

Example references to the evaluation findings . The following points 
suggest some of the evidence which gave rise to this conclusion. 

1. The survey shows that a large majority of project directors 
agree that efforts to win support for the project at their 
institution is important to project success. They also reported 
that leading causes of difficulty are communication problems 
within the institution. 

2. The survey also shows that conflicting commitments on the part 
of project staff present a serious difficulty. 

3. At Bay College lack of institutional support in the form of 
allocation of "real" release time had a severe impact on the 
ability of the project to continue past CAUSE funding. 

4. Hilltop University, Ivy University and Cedar State University 
provide examples of the key role institutional support plays in 
project success. 

Conclusion 2.7: CAUSE Projects Often Represent Innovations Within Their 
Respective Institutions Which Are Not Usually Innovations to Science 
Education . 

The large majority of project directors (94%) responding to the 
first survey reported that their projects were either very innovative 
(52%) or somewhat innovative (42%) as compared to the regular activities 
of the department(s) involved in CAUSE. Observations of the site visitors 
confirmed this report. Project directors are in general viewed as inno- 
vators within their departments and most often the project activities are 
unlike activities occurring in related departments. 

In general, however, the projects cannot be characterized as inno- 
vative with respect to science education as a whole since most of the 
projects we visited had counterparts occurring somewhere else in the 
country under the auspices of the CAUSE program. In fact, there are strong 
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similarities among projects even within the sample of 25 we observed. 
Although certain aspects of some of the projects do represent a unique 
approach or a unique application of a certain technology, there tends to 
be a substantial overlap among projects with respect to the problems 
grappled with and in the solutions found* 

This overlap is not always clear to project directors and staff. 
Most project directors reported on the survey cited above that they 
believe that their projects are in fact innovative with respect to 
science education. On the other hand, there is often relatively little 
awareness about what others are doing in related areas of instructional 
improvement. (There are a number of important exceptions to this, as 
some project directors and staff were extremely well informed,) Although 
there were many cases where it was obvious to site visitors that communi- 
cation among directors of similar projects would be particularly useful, 
relatively little such communication occurred. 

It is not necessary to be innovative nationally to be successful. 
It is not even necessary to be innovative locally if the needs are legi- 
timate and of high priority. It is important to recognize, however, that 
many projects that are innovative locally could benefit substantially 
from investigating the present and previous efforts of others attempting 
similar projects so that something might be learned from the successes 
and failures of others. 

Example references to the evaluation findings . The following points 
suggest some of the bases for arriving at this conclusion. Further evi- 
dence can be found in Volumes II and III. 

1. The survey shows a substantial number of project directors spon- 
taneously recommending more communication among project sites, 
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It is clear that they believe in the potential benefits of 
sharing information and "lessons learned' 1 . Fon;y-two percent 
of project directors report that science projects at other 
institutions similar to theirs have not been useful sources 
of information primarily because project directors have not 
located such projects elsewhere. 

2. At Blue Meadows State College an audio-tutorial approach to the 
teaching of science was regarded on campus (and locally) as a 
major innovation. 

3. Sands College and Marigold College provide examples of projects 
that were not innovative but which meet legitimate institutional 
needs. 

Conclusion 2.8: Comprehensive Projects Have Stronger Potential for Over- 
all Lasting Improvements to Undergraduate Science Education . 

In reviewing project implementation and impact at the 25 projects we 
visited, one overall theme emerged: the best projects were those which 
were most comprehensive. The CAUSE program guidelines encourage the con- 
duct of comprehensive projects. Such projects are described as "those 
which are broader and more integrated, and which include a set of 
coordinated activities". 

In practice, however, the concept of comprehensiveness has sometimes 
come to imply inclusivity, specifically in regard to the number of academi 
areas affected or involved. Some projects have faltered in their attempts 
to be comprehensive and, instead, have been somewhat scattered and dis- 
parate. Projects such as these are characterized by multiple and un- 
related mini-projects occurring where the staff of each remains relatively 
unaware of what the staffs of the others are doing. Although these indi- 
vidual projects have been sometimes of high quality, as a group they have 
lacked cohesiveness. 

A more appropriate approach to the concept of comprehensiveness would 
emphasize the importance of taking a holistic or systematic approach to 
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the identification of local science education needs and problems. Those 
projects which have considered all of the elements of an instructional 
system as it relates to some problem are the most truly comprehensive and 
the most outstanding projects we visited. These projects have been charac- 
terized by the development of a consensus among project staff regarding 
the needs to be served and the means to serve them. Careful attention has 
been paid to the relationship of the project to all facets of the insti- 
tution, and efforts have been made to coordinate the various project com- 
ponents with each other and with ongoing institutional plans and activi- 
ties. It is this integration with the institution which marks compre- 
hensive projects. 

Example references to the evaluation findings . Evidence presented 
in Volume II provides support for this conclusion such as the following 
points. 

1. Forestview College and Cedar State University provide examples 
of comprehensive projects. 

2. Springs University and Ivy University are examples of projects 
which are considerably less comprehensive. 

Issue Three 

To What Extent Is Improvement to the Quality of 
Instruction Occurring as a Result of CAUSE ? 

Conclusion 3.1: Science Education Resources Have Been Improved . 

C/JUSE projects generally have strengthened science education 
resources with respect to instructional materials, laboratory and 
instructional equipment, instructional facilities, and the development 
of faculty skills in instructiorially related areas. The improvements 
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wrought by the projects are likely to provide, in most cases, long-term 
benefits to institutions in terms of their capability to provide quality 
science programs. Particularly in those projects which have involved the 
acquisition of equipment and/or the acquisition/renovation of institu- 
tional facilities, more or let-?- permanent improvement in science education 
resources have been made. Of course, there are exceptions, especially in 
those cases where project activities have primarily consisted of "one 
time" occurrences and where there appears to be little institutional 
support for the continued pursuit of project goals. 

In addition to the obvious improvement in science education resources 
such as equipment, facilities and materials, the improvement of faculty 
skills should not be overlooked. Due to participation in project activi- 
ties, faculty development has occurred both in those projects which have 
primarily focused on faculty development and in those projects where 
faculty acquisition of new skills has not been a primary focus but a 
spin-off of project activities. Some of the improved skill areas have 
been in the systematic processes of instructional development and in the 
application of computer technology to instructional purposes. 

In some cases, the capital investment in science education which the 
CAUSE project represents has increased the visibility and viability of an 
institution's science education programs. This seems especially true at 
smaller institutions where resources of the magnitude required were not 
available from alternative .sources and science education needs were 
severe and basic in type (e.g., no adequate lab facilities}- In some 
cases, the award of the CAUSE grant itself was used as evidence of via- 
bility and in turn served as a catalyst for the acquisition of additional 



74 



resources for science education from other sources external to the 
institution. 

Both human and material resources in science have been demonstrably 
improved due to the CAUSE program. It is unlikely that these improvements 
could have taken place without CAUSE, 

Example references to the evaluation findings . The following points 
suggest some of the findings which led to this conclusion. See Volumes II 
and III for further details. 

1. The survey shows the improvement in faculty skills running 
through many nf the questions like a leitmotif. Project 
directors reported spontaneously that faculty have learned 
new skills and new competencies because of the CAUSE program. 

2. The survey also shows that 54% of project directors expect to 
acquire or upgrade equipment, materials, facilities, or com- 
puters through their CAUSE grant. 

3. The projects at Sands College and Forestview College provide 
examples of the improvement of instructional facilities. 

4. Sycamore CC, Sea University and Hilltop University are examples 
of projects focused specifically on faculty development. 

5. Cedar State University, Willows University and Clay College pro- 
vide examples of faculty development occurring as a spin-off of 
their projects. 

Conclusion 3.2: Institutional Capabilities for Self-Assessment, Manage- . 
ment and Evaluation Have Not Been Substantially Changed . 

One of the primary charges of the CAUSE program is to improve the 

capabilities of institutions to perform self-assessment, management, and 

evaluation. On the basis of the evidence collected during the course of 

this study, however, it appears that this goal has fallen short of the 

nark. This is not to say that the CAUSE projects have not had some 

impact on the capabilities of individuals within the institutions with 

respect to these areas. Indeed, many projects have provided a powerful 
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means of professional development for faculty (especially project direc- 
tors), particularly in the area of management. 

In some cases, the processes which occurred during the proposal 
development phase of projects to some extent have increased the ability 
of institutional staff to think through their needs and to prioritize 
them. It is difficult to judge, however, how general this effect is and, 
given the evidence we have gathered, we suspect that the effect has 
occurred only in a minority of the projects. Furthermore, it may well be 
the case that the institution's capability for self-assessment was high 
prior to the CAUSE grant in those cases where the proposal development 
process involved a true and thorough analysis of institutional needs. 

As discussed more fully in the section on Issue Four, evaluation 
activities on the CAUSE projects have generally been disappointing. 
Although there are exceptions, it is difficult to see how institutional 
capabilities in this area could have been improved given the generally 
inadequate and inappropriate attention given to evaluation in the projects 
themselves. 

Example references to the evaluation findings . The following points 
suggest some of the evidence which led to this conclusion. 

1. Saints University has a capability for self-assessment, manage- 
ment, and evaluation which was used to plan their project. 

2. Coastal University has a well-known center for evaluation on 
campus but center staff were not utilized on the CAUSE project, 
and capabilities for self-assessment and management were in 
place before CAUSE. 

3. At Maples CCC the skills of the project director may have been 
sharpened due to his participation in CAUSE but the institutional 
capabilities were not. 

4. Willows University provides an example of an improvement made to 
the institutional capacity for self-assessment. 
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Conclusion 3,3: The Quality of Science Instruction Has Been Improved 
Within Most Inst it utions Funded by CAUSE . 

J' This conclusion is based primarily on the judgment of the site 

visitors after consideration of the instrjction&l processes and products 
developed and being developed under the auspices of CAUSE. It relies 
only slightly on data directly reflecting chc^ges in student behavior, 
knowledge or attitudes since, as discussed in the following conclusion, 
there is very little of such data available. We also have relied c: sur 
vey data in which project directors report improvements in student per- 
formance attitudes. This, of course, cannot be considered as direct 
evidence since it is based on self-report. 

The strongest evidence for the improvement of the quality of instruc 
tion is the expansion of instructional opportunities for students. Many 
of these opportunities have been designed into courses as required acti- 
vities and many others have been designed to be supplementary to existing 
course materials and used at the students 1 option. In many cases the 
CAUSE projects have made available educational experiences that simply 
were not available previously (e.g., through the acquisition and develop- 
ment of laboratory facilities, both self-study and traditional, computer 
ha rr' 'are and software, revised curricula and the like). Although instruc 
tional materials prepared under CAUSE grants vary widely in terms of 
no mal instructional design criteria (clarity, organization, specif icatio 
of goals and objectives, readability, appropriate use of the medium, etc. 
most are being used by faculty and students who generally report them to 
be useful . 

Another sort of evidence of improved quality of instruction is based 
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Conclusion 3,3: The Quality of Science Instruction Has Been Improved 
Within Most Inst it utions Funded by CAUSE . 

This conclusion is based primarily on the judgment of the site 
visitors after consideration of the instrjction&l processes and products 
developed and being developed under the auspices of CAUSE. It relies 
only slightly on data directly reflecting chc^ges in student behavior, 
knowledge or attitudes since, as discussed in ths following conclusion, 
there is very little of such data available. We also have relied c: sur- 
vey data in which project directors report improvements in student per- 
formance attitudes. This, of course, cannot be considered as direct 
evidence since it is based on self-report. 

The strongest evidence for the improvement of the quality of instruc- 
tion is the expansion of instructional opportunities for students. Many 
of these opportunities have been designed into courses as required acti- 
vities and many others have been designed to be supplementary to existing 
course materials and used at the students 1 option. In many cases the 
CAUSE projects have made available educational experiences that simply 
were not available previously (e.g., through the acquisition and develop- 
ment of laboratory facilities, both self-study and traditional, computer 
ha rr' /are and software, revised curricula and the like). Although instruc- 
tional materials prepared under CAUSE grants vary widely in terms of 
no mal instructional design criteria (clarity, organization, specification 
of goals and objectives, readability, appropriate use of the medium, etc.), 
most are being used by faculty and students who generally report them to 
be useful . 

Another sort of evidence of improved quality of instruction is based 
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on the argument that providing support to qualified and dedicated science 
faculty to address local instructional needs will in itself lead to 
improved instruction, provided that resources and personnel are well 
managed and legitimate needs are being addressed. Whether the provision 
of support will actually result in improved instruction depends upon the 
skills and abilities of project staff. Project staff capabilities have 
been discussed in previous conclusions and have been found tc be generally 
quite high, notwithstanding the lack of certain specific areas of exper- 
tise previously discussed. 

A final evidence of instructional improvement may be in improved 
teaching effectiveness. A majority of project directors reported in the 
survey that faculty have improved their teaching and have developed better 
relationships with their students. Again, this is based on the perception 
of project directors and is not direct evidence of improved teaching 
effectiveness. 

Example references to the evaluation findings ., The following points 
suggest some of the specific evidence for this co J v . on. * 

1. An overwhelming majority of project direct ^s responding to the 
survey reported that students are performing better and have 
developed more positive attitudes toward science. 

2. Saints University, Coastal University, and Maples CCC are 
examples of projects where learning opportunities for students 
in required courses have been expanded. 

3. CCHEI and University ov the River are examples of projects which 
have improved supplemental instructional resources. 

Conclusion 3.4: The Direct Impact of CAUSE Projects on Students Cannot 
Generally Be Documented . 

Very little evidence exists at most CAUSE projects which shows a 

direct impact of project activities on student?' knowledge, attitudes and 
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behaviors. When such evidence does exist, as in the form of test scores 
or feedback on student questionnaires, it is difficult if not impossible 
to determine from available data whether or not the intact is more, less • 
or different from impacts that could have been expected from participation 
in non-CAUSE or pre-CAUSE educational experiences. 

There are a number of reasons for this lack of evidence. One reason 
is that most of the projects involve long-term efforts which are still in 
progress or have only recently ended. The real impact of many of these 
projects is yet to come. Another reason is that the relationship between 
many of the project activities and impact on students is necessarily 
indirect, particularly in the case of faculty development and facilities 
development efforts. In these cases the impact of the project on students 
must be ascertained through a logical analysis of ends and means as much 
as through the collection and analysis of empirical data. Finally, as 
will be discussed more completely in conclusions related to evaluation, 
the evaluation strategies used within CAUSE projects have generally been 
inadequate. The data collected has been insufficient to provide meaning- 
ful information on projects 1 impacts on students. 

There are a number of important exceptions to this conclusion, as a 
number of projects have systematically collected meaningful data relating 
to the impacts of project activities on students. However, these are in 
the minority and cannot provide a basis for reaching any conclusions 
about the CAUSE program in general. 

This conclusion should not be taken to imply that we believe C'.USE 
projects are not having a significant impact on students. We believe 
they are, as do the majority of faculty, staff and students interviewed 



79 



at the sites. However, our belief is based primarily on our subjective 
judgments and a reasoned analysis of the evidence collected during the 
site visits and reported in the site visit reports and case studies- 
Stronger empirical data could be obtained, in many cases, and would enable 
stronger conclusions to be made both within and across CAUSE programs. 

Example references to the evaluation findings . Some of the evidence 
which led to this conclusion is listed below. 

1. As was noted in the preceding conclusion, the survey data 
strongly supports the belief that there have been direct impacts 
on students zs a result of CAUSE activities but these data can- 
not be taken as conclusive since they are self-reported. 

2. Cedar State University and Saints University collected convincing 
data on their projects 1 impact on students. 

3. Sea University provides an example of maintaining computer usage 
data as indirect evidence of impact on students. 

4. Although there is little documentation of direct impact on stu- 
dents of the Forestview CAUSE project, it is cltiar to the 
observer and reader that important impacts will occur. 

Conclusion 3.5: A Range of Secondary and Unintended Effects Have Occurred 
As a Result of the CAUSE Projects . 

Several outcomes of the CAUSE projects have occurred which might best 
be classified as secondary or unintended since they are not within the 
scope of the original intentions or goals of the project. Nevertheless, 
these effect* probably have important implications for the quality of the 
science programs at the institutions at which they have occurred. 

One apparent and pervasive outcome, as evidenced by discussions with 
project staff during visits to sites and by a large minority of project 
directors reporting the outcome in an open-ended question in the survey, 
has been increased cooperation and communication among faculty. This is 
particularly true among faculty from different departments or divisions 
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who prior to the CAUSE project had little if any interaction. The CAUSE 
projects have apparently served as an opportunity for increased pro- 
fessional interaction among faculty. 

At several of the sites visited, staff reported that the CAUSE pro- 
ject has increased what had been a sagging staff morale. These reports 
are supported by open-ended and closed responses by project directors on 
the survey. Many project staff apparently have gained new self-confidence 
and self-esteem about their ability to have an impact. In some cases, the 
CAUSE project sparked a general revitalization of additional course and 
curricula which were not originally within the scope of project activities. 

In many cases, the impact of the CAUSE project has been felt in 
science areas not officially connected with the CAUSE project and in other 
non-science areas at the institution. In addition, it has been reported 
that in some cases the impact was felt outside the institution itself, 
either in the community at large or in other area educational institutions. 

Example references to the evaluation findings . The following points 
suggest some of the evidence for this conclusion. Further evidence can 
be found particularly in Volume II. 

1. At Forestview College the CAUSE project provided a major boost 

to sagging faculty morale at an institution in a state of crisis. 

2. At Central City JC, faculty and administrators from throughout 
the institution participated in inservice courses on instructional 
applications of computers as did facirUy from other educational 
institutions in the area. 

3. At Sea University the establishment of an instructional computing 
center led to an orientation program for all university students 
on how to access the computer. 

4. At Saints University a course redesign process was expanded and 
became much more comprehensive than originally proposed. 

~ 7 
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Issue Four 

What Is the Nature and the Quality of the Evidence 
and the Evidence Collection and Analysis Procedures 
Being Used to Determine the Strengths and Weaknesses 
of Individual CAUSE Projects? 

Conclusion 4,1: CAUSE Has Resulted in Increased Awareness of Evaluation 
in Undergraduate Science Education , 

A major goal of the CAUSE program is to "increase institutions' 
capability for self-assessment. . . and evaluation of their science pro- 
grams." The present study found that the CAUSE program has resulted in 
an increased awareness of the role of evaluation in improving under- 
graduate science education programs. While an increased awareness is 
short of an actual increase in capability and application, increased 
awareness is an important initial step to increased capability and usage. 

Most project directors and project staff interviewed professed to 
be ignorant of evaluation technology. Many spoke of having to go to the 
library to review the evaluation literature cited in the early solicita- 
tion or of having to ask the local evaluation expert to help respond to 
the RFP. Given that a large proportion of the proposal writers went 
through this procedure each year for the last four years, it is reasonable 
to say that awareness has been increased among this group. In addition, 
because project evaluation has been a component designed into each pro- 
ject it has had to be at least addressed with respect to the activities 
of the five to 20 faculty involved with each active project. Because it 
tended to be a new experience, and because some perceived it as a negative 
experience, CAUSE projects usually stimulated discussions about evaluation. 
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Finally, a few of the more successful projects published their results in 
professional journals. If nothing else, the CAUSE program has caused a 
larga number of science educators to grapple with the concept of 
evaluation. 

Example references to the evaluation findings . The following points 
suggest some of the bases for this conclusion. 

1. Eighty- five percent of project institutions agreed with the 
statement on the survey, "Evaluation plays a more important 
role in our CAUSE project than it does elsewhere in our 
institution's science programs. 11 

2. In the open-ended questions on the survey, several project 
directors used technical evaluation terms knowledgeably. 

3. Seventy-three percent of project directors reported on the sur- 
vey that "project staff have acquired additional expertise in 
evaluation as a result of the CAUSE project". 

4. At Bay College, the project director expressed great interest 
in discussing the place, role, and techniques of evaluation in 
science programs. 

5. The project director at Maples CCC talked to the director of 
institutional research at his college about evaluation and got 
help designing ar. evaluation plan, 

6. At Ivy University the project director designed a guide to 
evaluation in instructional improvement efforts and distributed 
it to-'f acuity in several departments. 

Conclusion 4.2: There Is Widespread Uncertainty Among Project Staff 
About the Nature and Purpose of Evaluatio n. 

While many project directors, in fact the vast majority, agree with 

statements from the survey like, "It is important that CAUSE guidelines 

require evaluation as part of projects", and "Evaluation is important at 

our institution in monitoring the effectiveness of projects of this type", 

there is a^o a strongly held feeling that precious project resources 

should not be used for evaluation when they are needed for implementation 

activities. The agreement with statements affirming the importance cf 
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evaluation suggests that some project directors are likely to verbally 
support evaluation while also feeling that evaluation is the least impor- 
tant of all project priorities. Consequently, there is a lack of clarity 
and focus in evaluation activities at CAUSE institutions. Sometimes the 
evaluation proposed initially has not been done or has been modified. 
Sometimes data are not utilized effectively. 

The site visitors, evaluators and science educators alike believe 
very strongly in the validity of this conclusion. While project personnel 
easily spoke about most of their project activities, interviewers found 
they had to probe substantially to understand the role of evaluation on 
the project. More often than not, the project director was the only per- 
son who knew what was happening in terms of evaluation of the project. 
Even <-,ien, questions were often deferred to "the evaluator". Evaluation 
activities were begun most often late in the project, sometimes not until 
the project was completed. Very few project staff ever gave the inter- 
viewers a sense that they had ever received any evaluation data that they 
found helpful, or that they were even curious about what evaluation data 
would have to say about their work. The apparent reason for this was 
that project staff saw little relationship between what they were trying 
to accomplish and project evaluation activities. 

The project directors 1 generally positive re, ird for evaluation and 
its utility as reported on the survey surprised us at first. Upon further 
reflection we believe it is not contradictory for two reasons. The first 
is that project directors as a group seem to be more conscious of the 
utility and worth of evaluation than their project staff. The second is 
that project directors 1 self- reported support for evaluation reflects an 



9n 



84 



acceptance of evaluation in principle that they actually find difficult 
to apply in practice. This difficulty, we believe, results from a limited 
understanding of the nature and purpose of evaluation, an understanding 
that emphasizes accountability to others somewhat to the exclusion of 
decision-making within the project. It is this limited understanding of 
the nature and purpose of evaluation that limits evaluation's utility to 
project staff in fulfilling their goals and objectives. 

example references to the evaluation findings . The following points 
represent some of the specific evidence which supports this conclusion. 
Further details can be found in Volumes II and III. 

1. In contrast to the agreement with positive statements jn the 
survey already noted, fully 25% of project directors also agree 
that evaluation activities probably require more time, money 
and effort than they are worth, and 27% agree that project 
funds allocated for evaluation activities could be better spent 
on other project activities. 

2. The survey shows that the most frequent aspects of the project 
to be evaluated are student reactions to the project, instruc- 
tional materials and student performance and that the most 
common types of evaluation data collected are multiple choice 
and essay examinations of student achievement and student 
opinions of project activities or outcomes. 

3. Project staff at Clay College and the project director at Blue 
Meadows State College claimed scientists really don't know too 
much about evaluation. 

4. The project director at Springs University reported that project 
staff were generally uncertain about "what NSF wanted" with 
respect to evaluation. 



Conclusion 4.3: Evaluations as Implemented Vary Considerably in Substance 
from Evaluations as Proposed . 

The majority of project directors responding to the survey reported 
that evaluation plans were being carried out as proposed. However, sur- 
vey results also showed evaluation to be one of the activities which, in 
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practice, most frequently deviated from the proposal's plans. This 
finding was generally confirmed during the site visits: evaluation plans 
were often being followed in name only with relatively little attention 
to the role evaluation activities could play in conducting or improving 
project activities. 

Almost all of the proposals dealt explicitly with evaluation. Some 
included quite detailed plans for quasi-experimental designs, followup 
studies and other data collection strategies. Others merely described 
who would be responsible for the evaluation. Almost all described evalu- 
ation as being an integral part of the project, many going so far as to 
include evaluation as one of the project's objectives. Some mentioned 
the use of formative evaluation to inform ongoing design decisions, but 
most stressed its role in determining project success. 

In practice _most of the more detailed evaluation plans were carried 
out less extensively than proposed, sometimes because the initial plans 
were not practical, other times because evaluation concerns were super- 
ceded by other project concerns. The summati ve-oriented evaluations were 
carried out but it was very difficult for site visitors to locate anyone 
in the project who seemed to be interested in the results, or who was 
waiting to make any sort of decision based on the data. Even in those 
cases in which numerous evaluation activities occurred, the resulting data 
were rarely referred to or discussed by project staff or administrators. 

Not all the variations from proposed evaluation plans were necessarily 
bad. Some projects increased or added formative evaluation activities to 
their original plans. In general, the added evaluation strategies which 
focused on improving developing products seemed to be more successful, 
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particularly when they were carried out in close coordination with the 
faculty involved in the development effort. In some cases a conscious 
decision was made by project staff to increase improvement-oriented 
evaluation activities. More often, these decisions simply manifested 
themselves in the day-to-day activities of the project. In a number of 
instances the conduct of activities designed to assess the quality of 
developing products had always been planned but had not been written into 
the proposal because it was not considered to be evaluation. In some of 
these cases the utility and effectiveness of these evaluation activities 
appeared to be quite high. 

The reason for the discrepancies between proposed and implemented 
evaluations seems clear. The CAUSE guidelines are quite explicit in their 
emphasis on the importance of evaluation. In order to respond, proposal 
writers typically went to the library or turned to their local evaluation 
consultant to help put together a credible evaluation plan. In this sense, 
evaluation plans were often prepared somewhat blindly and without careful 
tailoring to the proposed project activities, necessitating adjustments 
being made during project implementation. 

Example references to the evaluation findings . In addition to the 
survey results in Volume III, the reports of the projects at Coastal 
University, Willows University and Springs University in Volume II provide 
clear examples of projects whose actual evaluation activities differed in 
important ways from their original plans. 

Conclusion 4.4: CAUSE Has Had Little Impact on Institutions 1 Capabilities 
for Self-Assessment . 

Because evaluation has not played an important or prominent role in 

&3 



87 



the individual projects and has had little effect on them, it has not 
been possible for the CAUSE projects to have any real impact on an insti- 
tution's capability for self-assessment or evaluation of their science 
education programs. The project directors who already knew something 
about evaluation pretty murh adiirinistered the evaluation activities them- 
selves, resulting in no institutional change. Those who knew little about 
evaluation either retained the services of an evaluation consultant or an 
expert from a relevant science discipline from within the institution or 
from some other institution- In either of these cases very little inter- 
action occurred between evaluator and project staff which would increase 
the staff's capabilities in evaluation. In the case of evaluation con- 
sultants, efforts were focused on the collection of data. In the case of 
discipline experts, efforts focused on a review of course content. 

There were some important exceptions to this generalization. The 
increase in awareness of evaluation that was discussed in an earlier con- 
clusion is a step toward increased capabilities, although it is a rather 
small step. More specifically, a number of institutions did improve the 
evaluation capabilities of project staff and of the institution through 
the inauguration of evaluation procedures that did not previously exist. 
These projects provide useful models, but are not sufficient in number to 
be considered as evidence of the CAUSE program's impact on science 
education institutions' capabilities for self-assessment. 

Example references from the evaluation findings . The following 

points suggest some of the bases for this conclusion. 

1. Eighty-five percent of project directors reported on the survey 
that evaluation pays a more important role in their CAUSE pro- 
ject than it does elsewhere in the institution's science programs, 
while only 43% reported that "Our CAUSE project has helped 
science faculty members to integrate evaluation into ongoing 
science programs at our institution." 
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2. The Willows University CAUSE project has impacted on the Univer- 
sity's capacity for self-assessment in a meaningful way. This 
may also be true, to a lesser extent, at Cedar State University 
and at Hilltop University. 

Conclusion 4.5: Evaluation Has the Potential to Improve CAUSE Projects 
in Important Ways . 

In spite of the relatively ineffective role evaluation has played in 
most observed CAUSE projects, we feel that evaluation has an important 
role to play in future projects. The strongest evidence supporting this 
conclusion is the key role evaluation has played in selected areas of 
some of the more successful projects. 

The improvement-oriented or formative role of evaluation has the most 
potential for improving projects in that it relies heavily on the already 
existing expertise of project directors and staff and can {)ave a direct 
impact on the achievement of their own objectives. While there are speci- 
fic data collection and analysis techniques useful to formative evaluation, 
we saw several instances in which project staff with no training in 
evaluation were conducting extremely useful formative evaluations. The 
evaluation activities, sometimes not labeled as such, were characterized 
by explicit consideration of desired outcomes and careful observation 
of interim products (and the initial use of these products) to determine 
the extent to which the outcomes were being achieved. Sometimes these 
observations were formal and sometimes they were informal. However, 
they were always conscious and directed toward collecting information 
useful to the improvement of project activities or outcomes. 

There, were also examples of useful evaluations which focused on 
the quality and/or success of final products. While these types of 
evaluations were typically not very useful because of their lack of 
bearing on any real decisions to be made, there were some exceptions. 
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Some of these evaluations resulted in publications useful to others 
involved or considering involvement in similar activities; others were 
used to document the worth of the project to the institution in order to 
argue for post-project maintenance. Given that there are many projects 
attempting to accomplish similar objectives through similar means, it 
seems important to document the success of these methods to help others 
to avoid common problems. 

Another sort of evidence of the potential for evaluation of CAUSE 
projects was found at the many projects where evaluation could serve a 
useful role but did not. Many instructional improvements are being adopted 
with very little knowledge of their effectiveness. Many individual units 
or modules are being produced one after the other without any improvements 
being made in later products based on information about the earlier ones. 
This is clearly resulting in a lot of wasted effort and decreased overall 
efficiency (and probably quality) of product development efforts. 

The potential for evaluation is most likely to be achieved through 
the use of evaluation techniques and strategies which utilize the already 
existing observational and conceptual skills of practicing scientists and 
which are supported by the judicious use of professional evaluators who 
are willing to assist project staff in the design and implementation of 
evaluations which are useful and meaningful to the conduct of the projecc. 

Example references to the evaluation findings . The following points 

outline some of the specific evidence relevant to this conclusion. 

1. Saints University used informal and formal formative evaluation 
and did summative evaluation activities as planned. Maples CCC 
and the University of the River had formative evaluation acti- 
vities planned in thair proposals which they carried out. These 
activities appeared to be affecting the projects 1 outcomes. 
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2. The formative evaluation activities of the project staff at Elms 
College led to greatly improved audio-tutorial laboratory 
materials. 

3. The arrival of the formative evaluator at Willows University was 
termed a "turning point" by the project director. 

4. The instructional materials developed at Springs University could 
have benefited from a stronger formative evaluation component in 
the project. 

Issue Five 

What Are the Relative Costs of the Design, Implementation 
and Operation of Activities Within CAUSE Projects, and 
How Do They Relate to Post-CAUSE Institutional Support? 

Conclusion 5.1: Most CAUSE Projects Support Design and Implementation 
Expenditures for Instructional Improvements with Low Recurring Costs for 
Post-Grant Continuation of the Improvements . 

Overall we found that CAUSE funds generally have supported the design 
and investment expenditures necessary for capital improvement projects. 
Whether they have been primarily instructional development efforts or 
additions to equipment or facilities, most projects have represented a 
one-time allocation of resources to accomplish a specific kind of improve- 
ment that would continue to provide benefits in the future with little or 
no marginal recurring costs for their operation. 

Although this conclusion has grown out of the cost analyses of the 
eight case studies, the science educators and evaluators have come to 
similar, although less well -documented conclusions about the majority of 
the remaining 17 sites not visited by the cost analysts- In most cases 
the visitors have concluded that the improvements generated by the pro- 
jects would continue past the grant's expiration. An important factor 



#7 



91 



the site visitors took into account in judging the likelihood of con- 
tinuation was the demand for additional resources to support the improve- 
ment in post-grant years. 

This conclusion, in conjunction with the earlier conclusion that 
institutions cannot fund CAUSE-like projects on their own, points to the 
important function the CAUSE program has served in the improvement of 
undergraduate science education. CAUSE has provided higher education 
institutions with otherwise unavailable targeted funds necessary to bring 
about capital improvements in science education, improvements that once 
made can be continued with minimal drain on institutional resources. 

Example references to the evaluation findings . The reader is 

encouraged to read the cost anlysis sections of the case studies in 

Volume II. A few specific points follow. 

1- Saints University and CCHEI are examples of projects with one- 
time expenditures of resources for instructional development of 
redesigned courses and computing modules respectively. One com- 
ponent of the Saints 1 project was a capital investment in 
faci 1 i ties. 

2. Saints University's CAUSE project might have higher than average 
post-grant operating costs; however, it appears that these costs 
replace other recurring costs prior to the grant. 

3. Ivy University may face difficulties with post-grant continuation 
because of the relatively high recurring operation costs of its 
center for instructional development. 

4. Forestview College, Sands College and iMarigold College provide 
examples of projects in which the major focus was on capital 
improvement of facilities for science instruction. 

Conclusion 5.2: Institutional and Personal Contributions to CAUSE Projects 
Almost Always Exceed the Contributions Originally Proposed . 

In most cases the local CAUSE projects have served to coalesce and 

focus local resources for the improvement of science education beyond the 
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extent originally proposed. This was observed and documented most clearly 
in the cost analyses at the institutions described in the eight case 
studies, but it was frequently noted by site visitors to the remaining 17 
institutions as well. Sometimes the additional institutional contribu- 
tions have been quite obvious as when the institution covered the extra 
cost of additional or higher quality equipment than originally proposed. 
At other times these additional contributions have been less obvious, 
although just as real, as when the presence of the project has served as 
a means of attracting a variety of other institutional resources in the 
form of donated administrative time, or the use of additional university 
facilities or services for project-related purposes. The increase in 
visibility and reputation of a given department has also sometimes been 
reported as providing additional leverage to local science departments 
during institutional budget negotiations. 

Substantial personnel resources have also been donated to the pro- 
jects, most often by individual project faculty, in the form of evening 
and weekend hours. While at times the necessity for this extra time has 
been caused i>y inadequate planning or management, it often also has been 
the result of the motivational impact of a good project on the faculty. 
Good projects have generated excitement among faculty which has led in 
turn to a very strong personal and professional commitment among project 
staff to the project's goals and objectives. This capability of motiva- 
ting, exciting and focusing the efforts of faculty interested in 
increasing the quality of instruction is one of the important ways CAUSE 
has resulted in increased resources for the improvement of science 
education, resources beyond those initially proposed in terms of NSF and 
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institutional budgetary contributions. 

Ex ample references to the evaluation findings . The reader is 
encouraged to see the cost analysis sections of the case studies in 
Volume II for in-depth analyses of contributed resources. Some examples 
follow, 

1, Saints University and CCHEI are examples of projects which 
received large amounts of donated time from project personnel, 

2, Coastal University is a good example of a project which generated 
a sizable additional contribution from the institution for pro- 
ject activities, 

3, Cedar State University provides an example of how a CAUSE project 
can be used as leverage to procure additional institutional 
resources and as a means of capitalizing on existing university 
facilities. 

4, The existence of CAUSE projects at Fores tview College and Sea 
University was used as evidence of viability and commitment in 
obtaining additional funds from sources external to the 
institutions. 
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CHAPTER FOUR 
RECOMMENDATIONS 

Introduction 

A question asked of project directors and staff .during the course of 
the various site visits and also on the two surveys was "If you had one 
thing to recommend to the National Science Foundation regarding the CAUSE 
program, what would it be?". This question was often asked among the 
site visitors themselves during the course of the evaluation, and was 
asked again in writing of each of the site visitors before final deli- 
berations regarding the conclusions and recommendations, were conducted. 
Many more recommendations have been suggested over the course of the 
evaluation, most of which have been discussed or at least implied in the 
various levels of analysis included in Volumes II and III of this report. 
The recommendations presented here are those which emerged most frequently 
during these discussions and which appear to have the greatest potential 
for strengthening the CAUSE program. 

Recommendation One 
The CAUSE Program Should Be Continued and Strengthened 

The CAUSE program has been effective in improving the nation's 
science education resources. Institutional needs for improvement of 
instruction in science education are real and institutional budgets are 
unable to support the initiation of th'e sort of comprehensive projects 
brought about by the CAUSE program. Projects have been effectively 
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implemented and are meeting institutional needs as perceived by local 
science faculties. Through the CAUSE program's -requirement of matching 
funds as well as through the catalytic effect of many funded projects 
within their respective institutions, the CAUSE program has encouraged the 
targeting of institutional and human resources toward the improvement of 
undergraduate science education. The CAUSE program is effective. 

Although the basic organization and intent of the CAUSE program is 
sound and no significant weaknesses have been identified as inherent in 
its general approach, we believe that the CAUSE program can be streng- 
thened significantly through an increase in emphasis in a number of criti- 
cal areas. 

Recommendation Two 

The CAUSE Program Staff Should Be Enlarged, 
or Other Means of Providing Additional 
Personnel Support to the' CAUSE Program Should Be Provided 

There were very few complaints voiced by project directors about 
NSF's administration of the CAUSE program. Quite the contrary, comments 
on the surveys as well as those made during site visits were quite posi- 
tive. Although contact between CAUSE program staff and individual pro- 
jects was infrequent, the contacts that did occur were well regarded. 

However, it was clear from observations made during the site visits 
that a number of problems could have been avoided and projects often 
could have been more effectively implemented had the expertise of fellow 
scientists experienced in the improvement of instruction through large- 
scale development efforts been available. We feel the advice and support 
that is presently being given to project directors by program staff is 
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appropriate and useful and that an increase of such advice and support 
would result in a more effective CAUSE program. 



There are a number of specific areas of expertise useful to directors 
of large-scale instructional improvement projects. Comments made by pro- 
ject staff on survey forms and during the course of site visits as well 
as observations made by the site visitors themselves indicate that exper- 
tise in many areas is often unavailable to given projects. Specific types 
of expertise noted to be lacking most often include expertise in evalua- 
tion, instructional development, project management, hardware acquisition 
and instructional computer programming. 

Possible ways in which obtaining assistance in these areas could be 
encouraged, when the need exists, include suggesting the use of outside 
experts through program guidelines or during negotiations, supplying pro- 
ject directors with lists of sources of available expertise in each of 
these areas, and sponsoring targeted workshops for project directors 
needing assistance. 



Improvements in the communication and collaboration between CAUSE 
project directors could greatly improve the efficiency and effectiveness 



Recommendation Three 

Project Directors Should Be Encouraged and Assisted 
in Obtaining Specialized Expertise When Needed 



Recommendation Four 



Communications and Collaboration Across the Country 
Among Project Staff and Others Involved In and/or 
Experienced With Similar Instructional Improvement 
Efforts Should Be Encouraged and Supported 
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of many projects. Among the 25 projects visited as part of this study, many 
were quite similar in nature and intent. Many more similarities undoubtedly 
exist among all CAUSE-funded projects, not to mention other instructional 
improvement efforts not funded by CAUSE. As the site visitors went from 
project to project, they frequently saw project staff addressing and 
solving many of the same problems, but usually only after considerable 
time and effort had been expended. In those few projects where project 
staff had access to similar projects elsewhere, they seemed to benefit 
greatly from the experience of others. Unfortunately, most project 
directors are either unaware of other projects similar to their own, have 
little means of access to those projects, or do not understand how their 
project might benefit from another's experience. 

This situation exists in spite of efforts by the CAUSE program to 
disseminate information on funded projects and to bring the project 
directors together to meet one another. While these efforts are appro- 
priate, they are insufficient. The kind of information that most project 
directors would benefit from has more to do with the process of conducting 
project activities than with the products of those activities. New pro- 
ject directors need to hear the stories of how other projects have been 
conducted. They need to be able to talk to someone about the minutiae of 
project activities. 

It seems clear that the more that can be done to foster communication 
and collaboration among CAUSE project directors the better. In addition 
to the CAUSE program's existing efforts in this area, it is recommended 
that CAUSE encourage or foster further opportunities for project directors 
to meet and talk with one another. Specifically, the role of the present 
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project directors meeting should be clarified and extended. All project 
directors should attend one such "meeting a year. This meeting should be 
longer than it is at present, provide more opportunity for project direc- 
tor to project director interaction, and (to accommodate the needs of new 
projects) should be held in the summer rather than in the fall. The con- 
duct of such a meeting could represent an unrealistic burden for CAUSE 
program staff; however, the meeting could be organized and largely con- 
ducted by any of a number of external groups or organizations familiar 
with the CAUSE program. Much of the cost of these meetings would be 
underwritten by stipulated allocations in project budgets. Furthermore, 
it is recommended that proposal guidelines stress the desirability of 
establishing budget allocations for other kinds of project-related pro- 
fessional development activities. In addition to the project director 
meetings, most projects would benefit from opportunities for their 
faculty to attend workshops, professional association meetings and other 
events related to thei> projects' activities. 

A number of other steps could be taken as well to further project 
directors 1 access to information on the process of conducting CAUSE pro- 
jects. Careful descriptions of projects, such as the case studies 
presented in Volume II of this report, should be disseminated to the 
project directors. Proposals, interim reports and evaluation findings of 
selected projects could be gathered together in a kind of lending library 
for project directors. (It would also clarify the role of project 
interim reports and final reports if project directors understood that 
these reports were being written to inform future project directors.) 

Other ideas along these lines can be generated and these need more 
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refinement. The important thing to understand is that to be successful 
most CAUSE projects require some professional development of project 
staff in such areas as instructional development, instructional techno- 
logy, and so on. The CAUSE program should devote further attention to 
facilitating the professional development aspects of the projects in 
order to maximize their efficiency and effectiveness. 

Recommendation Five 

A Project Directors' Reference Manual Should Be 
Created to Provide Support and Direction to Project Staff, 
Particularly in the Areas of Project Management , 
Evaluation and Instructional Design 

There are a sufficient number of problems faced in common by most 
project directors and staff to justify the creation of a reference manual 
describing these potential problems and alternate means of attacking them. 
Included in such a manual would be descriptions of various management, 
evlauation and instructional design strategies used by successful project 
directors, and criteria to aid in the selection of the appropriate stra- 
tegy for a given situation. Of particular use would be hints, guidelines, 
and forms useful for the monitoring and control of project activities. 
The selection and use of outside experts would also be covered, as would 
be hints on the successful use of formative evaluation strategies. 
References to additional sources of information would also be included. 
The availability of such a manual, which would rely primarily on the 
experience of project directors of past CAUSE programs, would greatly 
decrease the present problems due to each project tending to start anew 
with little awareness of the progress already made by others. 
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Recommendation Six 

The CAUSE Program's Present Emphasis on Local Need and 
Institutional Support Should Be Continued, If Not Increaied 

i 

Effective CAUSE projects are based on high priority local needs. 
Although this statement is somewhat obvious, its implications for the 
planning and implementing of projects are not. It is relatively easy to 
construct a statement of institutional need that does not contradict the 
institution's general goals and which directly supports whatever instruc- 
tional improvement project a Creative faculty member would like to imple- 
ment. However, it is extremely difficult to identify, clarify and arti- 
culate a consensus on local instructional needs in a way that drives and 
focuses project planning, design, implementation and evaluation. However, 
without a clear, common and meaningful understanding of what needs are to 
be filled and exactly how project activities are to meet those needs, the 
filling of the needs will depend more on chance than on design. Because 
it is so easy for institutional administration and project staff to climb 
aboard the bandwagon of an intuitively appealing solution and so difficult 
to clearly and meaningfully articulate the concept of high priority insti- 
tutional need, it is critical -that project staff be encouraged and assisted 
with respect to recognizing the importance of local institutional needs as 
the driving force of their projects, 

A number of specific actions can be taken to make the present emphasis 
on institutional needs more meaningful to project planners and implementers. 
The statement of institutional support should represent the commitment not 
only of the institutions highest levels of administration, as is pre- 
sently the case, but of the department chair(s) and dean(s) levels as well. 
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This statement of commitment should explicitly define the specific needs 
to be addressed by the project and should describe the institution's 
specific commitment to the project in terms of how faculty time will be 
reallocated, what responsibilities participating faculty will be released 
from (and how those responsibilities will be covered), and the incentives 
the institution will provide participating faculty to enhance their con- 
tinued interest and participation in the project. Evidence of individual 
faculty members 1 personal and professional commitment to the project 
should also be emphasized in project proposals. 

Recommendation Seven 

A Greater Emphasis Must Be Placed on the Clarification 
and Improvement of Proposals Once the Proposal Review 
Process Is Completed but Before the Grants 
Are Formally Awarded 

Some amount of negotiation between , CAUSE program staff and selected 
grantees presently occurs as part of the formal award procedure and the 
modifications to original proposals recommended by proposal reviewers and 
CAUSE staff appear to have been generally sound. (Although pre-award 
negotiations were not a formal focus of our evaluation, documentation of 
these negotiations were often reviewed by site visitors.) However, we 
believe that it is this point in the project development process that NSF 
has the greatest power to positively affect the quality of CAUSE projects. 
Project staff will be more strongly motivated to put additional time and 
effort into the improvement of proposals because they know the probability 
of award is great. This is also an opportune time to require the institu- 
tion's administrators and department faculty to become more precise about 
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the nature of their commitment. In particular, it woyld be an ideal time 
to fully explicate the issues of release time, management and logistics, 
and evaluation. 

Since the recommendations of the proposal review teams are an impor- 
tant source of information for CAUSE program staff during the pre-award 
process, it is important that each review team include at least one person 
knowledgeable and experienced in comprehensive instructional improvement 
efforts in higher education. We believe a number of problems within 
specific projects could have been avoided had they been identified during 
the proposal review process. 

Recommendation Eight 

More Emphasis Should Be Placed on the Relationship 
Between Development and Operation 
Costs Within Proposed Projects 

The distinction between development costs (i.e., the costs required 
to bring about an instructional improvement) and recurring operation costs 
(i.e., the costs required to maintain an instructional improvement) is not 
usually addressed in project proposals, nor does it appear to be one 
consciously made by many project directors. However, the formal consi- 
deration of the cost structures of the eight case study sites as well as a 
more informal consideration of costs within each of the other sites 
visited strongly suggests that the distinction is extremely important in 
CAUSE, particularly witn respect to its bearing on the post-grant main- 
tenance of projects. The cost? of most of the projec :s visited were pri- 
marily development ccsts. We balieve that most of the instructional 
improvements brought about by these projects will continue to be maintained 
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by their institutions upon completion of the grant. The relatively high 
recurring operation costs of a few of the projects raised some doubts as 
to these projects' continued viability within their respective institu- 
tions in post-grant years. During the solicitation and award process an 
explicit emphasis should be placed on determining the proportion of the 
project which is a recurring operation cost. Thos^ projects with a high 
portion of recurring operation costs should be required to carefully 
justify the projects 1 plans in terms of long-term benefit to the institu- 
tion and/or maintenance of the projects 1 improvements. The commitment 
and ability of the institution to continue the proposed instructional 
improvement in post-grant years should be examined very closely by the 
proposal reviewers and the CAUSE program staff. 

Summary 

The CAUSE program is meeting extremely important institutional needs 
for improved science education resources. Some strengthening of, the 
program, as suggested by the above recommendations, would enhance the 
implementation and increase the impact of local projects. 
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BIOGRAPHICAL INFORMATION ON EVALUATION STAFF AND CONSULTANTS 



Thomas Allen is presently an Associate Professor of Chemistry 
at Hudson Valley Community College. Over the past several years he has 
created over 10 module programs for use in introductory chemistry courses 
at the two-year college level. These modules were developed for both 
computerized and audio-tutorial instruction. In an article entitled 
"Computers, Anyone", Mr. Allen discusses integrating computer technology 
with classroom instruction. He has had extensive teaching experience at 
the two-year college level, and has also consulted with many two-year 
colleges to determine educational needs and the appropriateness of inter- 
facing computer technology in instructional settings. Mr. Allen served 
as an evaluation team member, specializing in science education, on one 
site visit and one case study of the CAUSE evaluation. 



Albert Beilby is the Assistant tc the Director for the Office 
of Library Services at the State University of New York in Albany, New 
York. Dr. Beilby has been working on the development of simplified and 
generalizable techniques for the collection and analysis of functional 
cost data. He has conducted cost studies in a variety of educational 
settings including universities, two-year colleges, small liberal arts 
colleges and U.S. Army officer training schools. At the present time he 
is working on efficient, generalizable cost modeling procedures for use 
in evaluating innovative post-secondary programs. He is also analyzing 
State University of New York library holdings in order to identify factors 
to be considered in the future development of SUNY library collections. 
Dr. Beilby served as a cost analyst on three of the case studies in the 
evaluation of CAUSE. 



Jacquelyn L. Beyer is currently a professor i Yi the Department 
of Geography and Environmental Studies at the University of Colorado, 
Colorado Springs. Her past teaching experience is extensive including 
work at Rutgers University, the University of Cape Town, Antioch College, 
Montana State University, Columbia University, and the University of 
Texas. Dr. Beyer has also been involved in developing and implementing 
PSI in several content areas. Her concern for excellence in teaching and 
instruction has led to the development of many innovative programs. One 
of her current interests as a recently licensed private pilot is to 
develop an interdisciplinary course on "Science from the Window of an 
Airplane". Dr. Beyer served as a content expert in science education on 
two of the site visits and one of the case studies for the CAUSE evalua- 
tion. 



David Butts earned a Ph.D. from the University of Illinois in 
Science Education. He is currently Chairman and Professor of Science 
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Education at the University of Georgia. He has been the Director of the 
NSF Leadership Conference for College Educators (1969), the Director of 
the National Science Foundation Leadership Conference (1967), and has 
written 14 books and numerous articles in science and science education. 
He is actively involved in a variety of professional organizations. 
Dr. Butts participated as a science education consultant on one of the 
site visits and one of the case studies for the evaluation of the CAUSE 
Program. 



Jane G. Cashell has served as the project co-director of the CAUSE 
evaluation for Development and Evaluation Associates, Inc. In that position, 
she has assumed various responsibilities. She has served as an evaluation 
team member and reporter for three site visits and two case studies. Her 
other responsibilities have included design and analysis of the Survey of 
CAUSE Project Directors, analysis of funded proposals, development of site 
visit and case study methodology and reporting, and analysis of results 
from site visits, case studies, and all data collection activities. She 
has managed the staff, activities, and budget for the evaluation. 

For a DEA contract with the American Express Company she designed 
the evaluation component of a training system. Prior to working at DEA, 
she worked for two and a half years on the evaluation staff of the Center 
for Instructional Development at Syracuse University. Her next project, 
once the CAUSE evaluation is completed, is a conceptual analysis of some 
of the activities related to the activity of evaluating. 

Ms. Cashell holds a B.A. from Antioch College in elementary education 
and a M.S. in instructional technology from Syracuse University. She is 
a doctoral candidate in the Instructional Design, Development, and Evalu- 
ation program at Syracuse. Just recently Educational Leadership and the 
Journal of Research in Science Teaching have accepted for publication 
articles for which she is co-author. 



Terry Coleman , Research Associate with Development and Evalu- 
ation Associates, has served as a member of the primary evaluation staff 
on the study of the CAUSE program. His foremost responsibilities on the 
project have been to serve as evaluation team member for four site visits 
and two case studies. In addition, he has participated in staff deliber- 
ations regarding design, methodology, and analysis of data. 

Several of Dr. Coleman's recent projects have included the design 
of a nationwide evaluation of a school health curriculum for the National 
Center for Disease Control, the development of a training evaluation sys- 
tem for New York State's Department of Social Service, and the develop- 
ment of training materials in evaluation for a foreign governmental edu- 
cational television network. He holds a B.A. degree in English education 
from the State University of New York at Albany (SUNYA), an M.S. in Edu- 
cational Communications from SUNYA, and a Ph.D. in Instructional Design, 
Development, and Evaluation from Syracuse University. 
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Esther Lee Davenport is currently a Research Engineer at the 
Georgia Institute of Technology. She is working on a Ph.D. from the same 
institution in Industrial/Organizational Psychology. Her past experience 
is diverse and includes the development and management of a program for 
women in engineering, and participation in several NSF program evaluation 
studies. Ms. Davenport's current interests are in the areas of program 
evaluation and applied research on learning and training. Ms. Davenport 
served as an evaluation team member with expertise in science education 
on two of the site visits and one of the case studies. 



Philip L. Doughty conducted a cost analysis for the case study 
of Cedar State University in the CAUSE evaluation. He is currently an 
associate professor and chairman of the Instructional Design, Development 
and Evaluation program at Syracuse University. His graduate degrees and 
professional activities have emphasized the application of cost-effec- 
tiveness analysis and other management techniques to the design and 
operation of educational systems. In particular, he has designed and 
utilized cost-effectiveness models for teacher education, post-secondary 
education, and military training. Dr. Doughty has also designed and 
developed a computer-based test-development program and has engaged in 
numerous instructional design and development projects. He was project 
co-director on the development of evaluation protocols and procedures for 
biomedical communication directors under contract to the National Medical 
Audiovisual Center (NMAC) in Atlanta, and is currently developing the 
cost analysis procedures for the evaluation model for the New York State 
Department of Social Services. 



Marvin Druqer served as an evaluation team member on two site 
visits and one case study for the evaluation of CAUSE. He earned a Ph.D. 
from Columbia University in 1961. In addition to his scientific publi- 
cations, he has written and lectured extensively *on innovative techniques 
in science education. He has been the Chairman of the Education Committee 
of the Genetics Society of America for the past three years and was 
nominated for president of the National Association for Research in 
Science Teaching. He is the president. elect of the Society for College 
Science Teaching. Dr. Druger has participated in the Advisory Panel and 
conducted a portion of the site visits during Phase I of this evaluation. 



John D. Eqgert has served as the project co-director for the evalu- 
ation of CAUSE. He was responsible for the original design of the evaluation 
and, with Ms. Cashell, has overseen its implementation. He has served as an 
evaluation team member and reporter on seven site visits and three ca*e 
studies. Other responsibilities have included participation in the design 
of the site visit and case study methodology and in the analysis of the 
data across cases and sites, and across all data sources. 

Dr. Eggert is Director of Development and Evaluation Associates, Inc. 
(DEA). Projects which he has co-directed include a feasibility study and 
plan for the evaluation of the Center for Disease Control's School Health 



Curriculum Project and the design for a state-wide evaluation system for 
Title XX training for the New York Department of Social Services. He 
has also participated in a major staff development effort for the Govern- 
ment of Iran's educational radio and television network. Prior to his 
tenure as DEA's Director he served in the evaluation office of the Uni- 
versity of Mid-America and also designed and implemented an evaluation 
of a Training of Teacher Trainers (TTT) project at the University of 
Wisconsin-Milwaukee. John received his doctorate from the University 
of Chicago's MESA program (Measurement, Evaluation and Statistical Analy- 
sis). In his dissertation he developed a model for emotional response 
to music and investigated that model in an academic setting. 



James J. Gallagher received an Ed.D. in Science Education from 
Harvard University. He is presently Director of the Michigan State Uni- 
versity Science and Mathematics Teaching Center. He has written exten- 
sively in the field of science education and has served as President of 
the National Association for Environmental Education Committee and as 
Director of Education for the American Society for Environmental Educa- 
tion. Dr. Gallagher is now president-elect of the Michigan Association 
of Science Education Specialists. During the CAUSE evaluation, 
Dr. Gallagher participated as the science education consultant for two 
of the site visits and one case study. 



Rame'sh Gaonkar served as an evaluation team member specializing 
in science education on one case study and one site visit conducted for 
the evaluation of CAUSE. He is a professor at Onondaga Community College, 
Syracuse, Mew York. He earned his Ph.D. from Syracuse University in 1974 
through an interdisciplinary program in Instructional Technology in the 
School of Education and Electrical Engineering in the School of Engineer- 
ing. He has taught physics in both the United States and in India. Dr. 
Gaonkar has participated in numerous curriculum development and evaluation 
projects and has presented several seminars and guest lectures. 



Richard M. Lent is a research associate at Development and 
Evaluation Associates, Syracuse, New York. Dr. Lent's specialty is in 
the use of cost-effectiveness analysis as a means of planning and evalu- 
ating projects involving the use of instructional development and educa- 
tional media. He has conducted a number of studies in this area and has 
written and taught on the subject both here and abroad. He has also 
served as a consultant on curriculum and instructional development pro- 
jects in a variety of traditional and non-traditional post-secondary 
education settings. Dr. Lent came to DEA from the Sta.te University of 
New York where he was assisting one of its institutions in making the 
transition from a four-year to upper-division only college. He received 
his Ph.D. from Syracuse University in Instructional Technology. 

Dr. Lent participated in a number of activities for the 
evaluation of CAUSE including the Forestview Case Study, three site 
visits, and three cost analyses. He also was responsible for managing 
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the cost analysis team and had major responsibility for analysis of 
findings from all the case studies and site visits. His other projects 
at DEA have involved the creation of a training system for American 
Express and the design and conduct of staff development workshops for 
state and federal agency training departments. 



John E. Penick earned his Ph.D. from Florida State University 
in science education. He is presently an associate professor at the 
University of Iowa. He has conducted a number of research studies in 
science education and is a member of a variety of professional groups 
including the Association for the Education of Teachers in Science, the 
American Educational Research Association, and the National Association 
for Research on Science Teaching. He has made many presentations and 
has conducted numerous workshops for school districts around the country. 
Dr. Penick served as an evaluation team member specializing in science 
education for one of the case studies and one site visit. 



Kathleen Porter served as a graduate intern on the CAUSE evalu- 
ation project for Development and Evaluation Associates, Inc. Her respon 
si bi 1 i ties on the project centered on the broad focus evaluation activi- 
ties, the content analysis of funded proposals and the surveys of project 
directors. She was involved in instrument development, data collection, 
data analysis and reporting. 

Now, as a research associate with DEA, she is working on the develop 
ment of an evaluation system for the Department of Social Services in the 
State of New York. Sire is primarily interested in evaluation in higher 
education settings and has much background in faculty development. 

Dr. Porter earned her Ph.D. in English from Syracuse University 
and was an assistant professor of English at St. Lawrence University. 
She was responsible for editing the complete final report of the evalua- 
tion of CAUSE. 



Peter A. Stace earned his doctorate in the Area of Higher 
Education and his masters degree in economics at Syracuse University. 
Presently, Dr. Stace is an Assistant Dean in the College of Arts and 
Sciences at Syracuse University where his major responsibility is to 
direct undergraduate services and advising in the College. A primary 
area of interest for Dr. Stace is that of cost-effectiveness analysis 
and its application as a tool for program evaluation. He has completed 
his dissertation and several other studies in this area and has conducted 
workshops on the topic. Dr. Stace served as a member of Development and 
Evaluation Associates' cost analysis team during the CAUSE evaluation and 
conducted the cost analysis for the case study of Willows University's 
CAUSE project. 
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Spencer Swinton served as an evaluation team member on one of 
the site visits conducted as a part of the CAUSE evaluation. He completed 
his undergraduate education at MIT and earned a Ph.D. from the University 
of Chicago in 1973 in the area of Measurement, Evaluation and Statistical 
Analysis. Presently, he is a research psychologist in the Division of 
Educational Research and Evaluation at the Educational Testing Service. 
He has been chairman of the Department of Mathematics at the Government 
Teachers' College, Dar es Salaam, Tanzania and an Assistant Professor of 
Child Development at the University of Wisconsin. Dr. Swinton served as 
the project director for NSF's PLATO Elementary Computer-Based Education 
Evaluation and has had experience in a variety of research, development, 
and evaluation projects. In addition to publications in the areas of 
cognitive development and evaluation, he holds a recent patent in the 
field of educational technology. 



Jody Karen Witham served as a graduate intern on the CAUSE 
evaluation project for Development and Evaluation Associates, Inc. Her 
responsibilities on the project centered on the broad focus evaluation 
activities, the content analysis of funded proposals and the surveys of 
project directors. She was involved in instrument design and development, 
data collection, data analysis and reporting. 

Ms. Witham holds a B.A. in Psychology and an M.A. in Education, 
both from the University of Vermont. She is a doctoral candidate in the 
Instructional Design, Development and Evaluation program at Syracuse 
University. She is primarily interested in "regular" education for 
"special" children, and the development of principles of motivation 
which can be applied in the design of instruction and classroom 
management. 



Robert E. Yager earned his Ph.D. from the University of Iowa 
in 1957. He is presently director of the Science Education Center at the 
University of Iowa. He has directed over 70 different NSF sponsored 
programs and institutes, many of which involved the evaluation of inno- 
vative science education programs and activities. He has served as a 
consultant and an evaluator for proposals, accreditation, project evalu- 
ation, and synthesis studies. He has been president of a number of 
professional organizations including the National Association for Research 
in Science Teaching and the Association for the Education of Teachers 
in Science. Dr. Yager served on the Advisory Panel during Phase I of 
this evaluation. He also participated as an evaluation team member 
specializing in science education for four of the site visits conducted 
during the CAUSE evaluation. 
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